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L7: Entry 7 of 112 File: USPT Jul 31, 2001 

DOCUMENT- IDENTIFIER: US 6269356 Bl 

TITLE: Computer system program for creating new ideas and solving problems 
ABPL : 

A computerized idea generator and problem solver that can receive English 
language sentences describing a problem to be solved and generate solutions by 
analogy. The computer uses words in the problem description sentences to obtain 
information from a database that relates to the type of problem to be solved, for 
example, animal, plant or human. The computer then uses this information to 
access analogous information from a second database. The information used can 
include (1) where a noun in the problem description is categorized in a hierarchy 
of information within the database, and (2) what purpose or what function is 
provided by that noun. This information is used to obtain a noun from a second 
database and a description of how to solve an analogous problem with respect to 
that noun. The original noun is substituted in the sentence for the second noun 
to arrive at a solution. 

BSPR: 

Expert systems are computer systems that are used to store large amounts of 
information on a particular topic in a computer database and are designed to 
respond to queries from a user to provide information about a particular problem. 
For example, there are expert systems that are used to guide boats through a 
harbor. Such an expert system contains information in a database that relates to 
the heights of bridges and the depths of channels in the harbor. A user can query 
this system on what route it should take with a boat having certain dimensions. 

BSPR: 

In particular, in one embodiment of the invention, a computer program receives an 
input statement including a description of the problem and a goal statement. For 
example, "The beavers destroyed the bark. Save the tree." The computer program 
determines that the word "bark" represents an object of the verb repair, which is 
associated with the word "save" in the problem statement. The computer program 
then looks up the work "bark" in the tree Experience Database. The program 
determines that the "bark" entry in the tree Experience Database is at the second 
hierarchical level in the data base and that its function is to "protect" the 
tree. The program also finds that the tree Experience Database has no "repair" 
function for bark. The computer program then consults a second data base, for 
example the human Experience Database. It looks for an entry at the second 
hierarchical level whose function is to "protect" the human, which is the entry 
corresponding to "skin" . The computer program locates the entry in the human 
Experience Database that corresponds to the repair function for skin: "Transplant 
skin to damaged location." The program then substitutes the word "bark" for the 
word "skin" in the above sentence to result in a solution to the problem: 
"Transplant bark to damaged location." This solution can then be stored in the 
tree Experience Database so that if the same problem arises again the computer 
program will not have to create the solution a second time. 

BSPR ' 

This' invention is a high level computer program that can create new ideas and 
solve a broad range of problems. It does this through the word analysis of a user 
inputted problem statement (an English sentence) and a goal declaration (most 
often a verb phrase) . The computer program analyzes the problem statement and 
goal declaration looking for key words that fall into the classification of: an 
Action Maker (subject); an Action (primary verb); and Action Taker (ob:ect) . Each 
word (either subject or object) is referenced to dictionary data which has been 
previously categorized. For example, the word bark is in the plant category and 
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further divided into the sub category of tree. 



BSPR: 

A section in the computer program looks at the input information (problem 
statement and goal declaration) to see if it fits the logic of the world in terms 
of create and destroys since most things in the world eventually fit into these 
categories--things are created and then, over time, erode back to the material 
that created them. 

BSPR: 

This invention provides a system to emulate human creativity and the human 
problem solving process using a defined "solution processor" as disclosed in this 
application, and "Experience Databases." An "Experience Database" is a 
description of any system in the universe formatted as a hierarchical data 
structure. The tree system (called the tree Experience Database) is shown in FIG. 
1 and 3 in this document. FIG. 3 is a partial listing for category "plant" and 
subcategory tree. It defines the functions of a tree starting at level one. Tree 
has four level one functions which are all defined as the first argument of one, 
namely, branch, trunk, root, and nutrient. The process "branch", for example, 
performs the function [hold, branch, leaf ] . The verb "hold" relates branch and 
leaf. It is read as "branch hold leaf." Any analogy search would look at selected 
"holds" (based on program logic analysis) to determine if the arguments of this 
"hold" could be substituted to solve a problem or create a new idea. FIG. 1 
details the same information as in FIG. 3, but is shown in a pictorial format. 
The "Experience Databases" act like real human experience and allow the system 
disclosed to act in a way that, by adding more "Experience Databases," the system 
increases its ability to suggest ideas and find solutions. 



DEPR: 

The key component to this general purpose problem solving system lies in the 
"Experience Databases". (See FIGS. 1 and 2.) This is new and novel and ties 
together this system's unique ability to generate ideas or solve any problem 
using these database structures and the search processes employed in this 
software program. From the logical analysis of the input statements, the program 
decides which "Experience Database" to search to find its answers. The search 
mechanism within this software program must have databases organized in such a 
way that it lends itself to the selection of the appropriate answer (s) . The 
database is organized based on actions or verbs that tie all the nouns together 
in the world - this tying together of all the nouns by verbs is done by a 
hierarchical system of databases for each category. For example, tree is a sub 
category of category plant and the beaver tree problem is solved, in part, by 
using the tree.exp database. What causes analogy to serve as a basis for problem 
solving is predicated on the fact that all systems in the universe form under 
some sort of force. Across each system as in plants and animals is the common 
forces of nature. In the case of plants and animals, as one example, you would 
have the force that causes each system to protect (verb) itself against the force 
of weather. The protective device is bark and skin respectively. In the case of 
the "beaver tree problem" (as it will be referred to throughout this document) 
each system, plant and animals alike, are forced to develop similarly. These 
similarities are apparent if one were to break down plants and animals (m this 
case) in a hierarchical structure using functions and processes. In FIG. 
l--tree exp, a tree has a function called " support ( trunk, bark) ." It is made of a 
process called "protect (bark, tree) " and others that are not shown. Likewise, in 
FIG 2--human.exp, a human has a function called "perf orm { organ, function) " and 
has as one of its processes a function called "protect (skin, human) These two 
functions: "protect (bark, tree) " and "protect ( skin, human) " occur at level 2 m the 
hierarchy for both the ( "Tree . exp" , plant) and ("human. exp" , animal") "Experience 
Databases." The format for any level in an "Experience Database" is: 



DEPR * 

In summary and up to this point, we were given the information that a beaver 
destroyed (removed) the bark from around the base of a tree. If gone untreated 
the tree would die. An idea was tried and strips of bark (grafting) were placed 
across the area of removed bark which proved to be a method to save the tree from 
dying The method by which this problem was solved, using some theories on the 
structures of systems and the way the human brain may solve problems, was 
implemented in a computer software program. Part of the way the software program 
goes about solving this problem is by analysis of the problem statement (The 
beaver destroyed the bark) and the goal declaration statement (Save the tree) . 
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There is a lot of sentence processing by the software program, but it eventually 
leads to the "Experience Databases" for plants and animals. It focuses in on two 
key parts of the Experience data and starts by isolating this information in the 
plant "Experience Database" (tree.exp) and homing in on the function identified 
as "protect (bark, tree) . " It then looks for the "protect" verb in the (human. exp) 
database and finds "protect ( skin, human) " noting that this also occurs at Level 2 
in the hierarchy. (See FIGS. 1 and 2.) The software program will now assume that 
"bark" and "skin" are analogous (because of the pointer verb "protect" and 
because they are both at the same level in their respective "Experience 
Databases") and begin a search in the (human. exp) database looking for a repair 
function for skin (because the software program knows bark was destroyed and it 
must be repaired by the goal declaration: Save the tree) . (Also note that the 
software program knows that "bark" in the problem statement and "tree" in the 
goal declaration come for the same category "plant" as defined in dictionary of 
this system.) When it finds a repair function in the "human. exp" database for 
skin (see FIG. 4B--data item starting with three ( [repair , skin . . . , it will 
substitute bark for skin (because of the common verb "protect") and solves the 
"beaver tree problem" by suggesting: 

DEPR: 

Note that the word beaver in item 7C above has been changed from cn to cnp . Since 
all words in the dictionary are singular (nouns) , beaver was changed to cnp 
(common noun plural) to reflect the word "beavers" in the problem statement -- see 
line item lA above. The dictionary words have three main purposes in solving 
problems. First, it is important for the system to identify each word Category. 
For example, the word bark is identified as Category "plant" as defined by the 
dictionary--see 4C above. The system must know this because it will go to the 
("tree.exp", plant) "Experience database" (FIG . 3) and begin to look through the 
hierarchical statements beginning at level 0 (first entry is: zero ( [environment , 
tree] , [0] , [living, physical , organism] ) looking for the word "bark" where 
"physical" is now. This does occur within (FIG. 3) at level 2 and is identified 
as : 

DEPR: 

The problem and goal declaration statements are broken down into Action_Maker , 
Action, and Action_Taker (see 26A, 27A, 28A, 29A, and 30A above) by the computer 
program. These then become the lowest common denominator in all problem and goal 
declaration statements, and allow the computer program to process similar problem 
and goal declaration statements in the same fashion so as to guarantee the same 
results (answers) , It further identifies who (noun subject) caused the action and 
who (noun object) received the action. Time and state verbs are discounted as not 
having a legitimate place in the interaction between nouns because its more 
difficult to describe any changes to noun subject or object. It is certainly 
possible to include rules (in the current computer program) that would account 
for any effect they may have on nouns. In general, adjectives and adverbs are 
also currently discounted, but in a rule based system, as this one is, it's 
relatively easy to add rules to address their effect on nouns, pronouns, and 
verbs . 

DEPR ' 

The "semantic logic system" transfers two important pieces of information to the 
"database search and logic system." The first is the "search list" and the second 
is the "seed word" based on the semantic analysis of the problem and goal 
declaration statements for the "beaver tree problem." The "database search and 
logic system" isolates the word "bark" knowing that the "semantic logic system" 
defined it as the Action_Taker in the problem statement based on analysis of the 
problem and goal declaration statements. The "database search and logic system" 
begins a search through the "tree.exp" database looking for the bark function and 
finds "protect (bark, tree)". (See FIG. 1 . ) . It then stores the list of words 
(protect, bark, tree] and uses the word "protect" as the analogy word for the next 
database search. The next database search has been defined by the "semantic logic 
system" in its analysis of the "beaver tree problem." It passes the "search list 
that contains its guesses of other "Experience Databases" to search for the 
analogy to "protect." It begins its search in the "human. exp" (experience) 
database looking at level 2 in the hierarchy and finds "protect { skm, human) ." At 
this point, the computer program displays the following information to the user 
on the CRT; 

DEPR: 
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Based on prior analysis by the "semantic logic system" the "database search and 
logic system" looks for a "skin repair" function within the "human. exp" database 
that relates skin and repair. It finds one (See FIG. 4B level 3) : 

DEPL: 

Level in hierarchy ( [Child , Parent ] , [Numeric Level in 
hierarchy] , [Verb, Noun_Subject, Noun_Ob j ect] ) . So for example, 
"protect (bark, tree) " can be read as 

"protect (Verb) (bark (Noun_Subj ect) , tree (Noun_Obj ect) )" and "protect ( skin, human) " 
as "protect (Verb) ( skin (Noun_Subj ect ) , human_ob j ect ) ) . These can be rewritten to 
appear more like English sentences as in: "bark protect tree" and "skin protect 
human." These "Experience Databases" were devised in such a way as to allow the 
software program to search for answers based on the organization of systems and 
their likeness to each other. The theory states that if you break up systems in a 
hierarchical format, as functions and processes, you will find the same verbs at 
or near the same level in the hierarchy across different "Experience Databases." 
What this further shows is that the verbs, in this case "protect", can be found 
in both the "tree.exp" and "human. exp" databases at level 2. This then being the 
case, it could be said that nature has made nouns (bark and skin) analogous to 
each' other because they are tied together by the common verb protect under the 
common force of weather. It would further suggest that a solution in a particular 
system may be applied to another system through its analogies. These analogy 
substitutions are not just limited to verbs like "protect" being at the same 
level in a hierarchy, as in FIGS. 1, 2, but also for "protects" that may occur at 
different levels or within other categories because nature tends to also 
duplicate functions at different levels across or within the same system as in 
the plant and animal systems. Note that "protect" is a function and could have 
many different arguments (nouns). It may be that "protect (paint , car) " , in 
addition to "protect (bark, tree) " and "protect ( skin, human) " , could be used to 
solve the "beaver tree problem." What "protects" that gets used by the system to 
solve problems is determined by the analysis of the problem statement and goal 
declaration statement. 



9 The system of claim 8, further comprising means for determining the 
hierarchical level of said entry in said first data base _ and wherein said means 
for obtaining data from said second data base obtains said data at said 
hierarchical level in said second data base. 
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hi: Entry 9 of 112 File: USPT Jun 26, 2001 

DOCUMENT- IDENTIFIER: US 6253064 Bl 

TITLE: Terminal based traffic management and security surveillance system for 
aircraft and other commercial vehicles 

ABPL: . ^ , . _ 

A traffic management, security and surveillance system for commercial vehicles 
while in port incorporates a plurality of strategically located sensors for 
identifying the location of both commercial vehicles and support assets, as well 
as personnel, for monitoring traffic flow and supporting traffic management of 
the commercial vehicles, assets and personnel while in the area. The location 
signals are transmitted to a ground based monitoring and control center and may 
be archived for later playback. The system also is adapted for utilizing on board 
location signal generators such as GPS sensors to provide traffic flow 
information and traffic management of all commercial vehicles, assets and 
personnel on the system . 

This application is a related to my copending applications entitled: Wireless 
Transducer Data Capture and Retrieval System for Aircraft, Ser. No. 08/745,536, 
filed on Nov. 12,1996/ Video and Data Capture Retrieval Surveillance System for 
Aircraft, U.S. Ser. No. 08/729,139, filed on Oct. 11, 1996; and Acoustic 
Catastrophic Event Detection and Data Capture and Retrieval System for Aircraft, 
U S. Ser. No. 08/738,487, filed on Oct. 28, 1996 now U.S. Pat. No. 5,798,458, and 
my copending applications Ground Based Security Surveillance System for Aircraft 
and Other Commerical Vehicles ; Apparatus for and Method of Collecting and 
Distributing Event Data to Strategic Security Personnel and Response Vehicles; 
Ground Link with On-Board Security Surveillance System for Aircraft and Other 
Commercial Vehicles ; and, Network Communication Techniques for Security 
Surveillance and Safety System, filed on even date herewith. 

BSPR * 

The subject invention is generally related to safety and surveillance equipment 
for airc raft, ships and other commercial vehicles and is specifically directed to 
a comprehensive multi-media traffic management and security surveillance system 
for same while in port or terminal and/or unattended whether taxiing or parked. 
The system of the subject invention also permits tracking while en route. 

BS PR * 

Security is of ever increasing importance. This is particularly true with respect 
to commercial airlines as more and more people and freight are moved in this 
manner and aircraft and other commercial transports increasingly become the 
targets of terrorists activities. The airways are becoming increasingly crowded 
with traffic. Global tracking systems are now in place to monitor the flight ot 
the aircraft from the moment it lifts off until it safely lands at its 
destination. Radar and navigational positioning systems are commonplace both on 
the aircraft and at the ground tracking stations. All of these electronic systems 
have increased the overall safety record of commercial traffic to new standards 
as the number of miles flown continues to escalate. 

BSPR * 

In addition, the on board avionics including electronic monitoring and diagnostic 
equipment, particularly on large commercial jets, continues to evolve, giving 
both the on board crew and the ground assets more complete, accurate and to 
date information regarding the condition of the aircraft while in flight . Flight 
recorders long have been incorporated in order to provide a record of each flight 
and in order to provide critical information to aid in the determination of the 
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causes of an accident or mult if inction should one occur. 
BSPR: 

However, one area which has been neglected with the ever increasing availability 
of electronic surveillance is the security of the aircraft or other vehicles or 
vessels, including, but not limited to, over- the-road vehicles, ships and other 
commercial transports (collectively referred to as commercial transports) , 
particularly when unattended. Typically, when an aircraft is on the ground, or in 
port, and unattended the only security provided is the security of the location. 
If the security of the area in which the commercial transport is stored is 
breached, the commercial transport is an easy target. In most cases, even the 
access doors are left open and further, for obvious safety reasons, are designed 
not to be locked from the outside. Many critical areas of the commercial 
transport are left exposed such as in an aircraft, by way of example, the landing 
gear, the engine housing and critical wing and tail components. 

BSPR: 

In addition, most of the ground traffic control relies on pilot visual 
information and onboard avionics providing an in adequate, at best, traffic 
management system utilizing already overworked equipment. The limited control and 
management of terminal traffic including not only the commercial transports, but 
also ground personnel, security vehicles and fuel trucks and other support 
vehicles, creates a dangerous and hazardous condition at most highly congested 
terminals . There are not any monitoring devices to assure that such traffic is in 
its assigned zones or to alert when a commercial transport is at risk because of 
breach of a restricted zone by unauthorized personnel or unauthorized vehicles, 
including overlapping exclusive occupancy zones by more than one commercial 
transport . 

BSPR: . . 

The subject invention is directed to a comprehensive multi-media safety, tracking 
and/or surveillance system, which in the preferred form provides both visual 
and/or audio information as well as critical data such as location, direction, 
intrusion, fire and/or smoke detection and/or status of environmental conditions 
and/or transport systems status. In my aforementioned patent and copending 
applications, incorporated herein by reference, detection and sensor systems are 
utilized to provide the flight crew and/or a ground tracking station for 
commercial aircraft critical information during flight and/or to record the 
information and data generated during flight for later reconstruction of 
catastrophic events. The subject invention is an expansion of this concept and 
adds not only ground security and surveillance, but tracking while in port or on 
the ground as well as while in route, as well as incorporating the onboard 
systems of the aforementioned patent and applications. It is an important feature 
of the invention that the transmitting network for provides a comprehensive 
communications link between stationary and mobile stations on the ground, the 
craft or vehicle being monitored and strategic sensors both onboard the 
commercial transport and the sensors on the ground. In the preferred embodiment 
of the invention, a wireless LAN (local area network), WAN (wide area network) or 
other wireless transmission scheme is used as the transmission system of choice. 
A digital wireless voice intercom is provided for security purposes and for 
communication between the onboard crew and the ground ^?^_P®_^f ^; 

ess 

(such as, by way of example, LAN) based file communication capaDiiity permits the 
transmission of information such as route or flight plans or gate and dock 
information. As an example, a LAN or WAN has worldwide tracking capability 
adapted to be used in connection with a global satellite communication system 
such as IRIDIUM, wherein the entire path and status of the commercial transport 
may be monitored while airborne over satellite connections. While wireless 
systems provide the preferred form of communication, many features of the 
invention may be practiced using other communication links within the scope and 
spirit of the invention. 

BS PR * 

One important feature of the invention is the ability to remotely monitor a 
commercial transport while on the ground, whether or not the commercial transport 
is attended. This will permit detection of unexpected events, breach of security, 
change in environmental conditions and other activities both on and in the 
vicinity of the commercial transport. A GPS system may be included to provide 
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accurate positioning information of the commercial transport, establishing the 
parked position of the commercial transport at any time, as well as tracking its 
movements . 

BS PR * 

Once in port, the system of the subject invention permits complete monitoring of 
on ground movement, and allows the monitoring of other commercxal transport in 
the area to assure that the various transports do not interfere with one another . 
This provides collision avoidance, and can be utilized both on the ground and in 
the air or in route via water or land. Current airborne collision avoidance is 
accomplished by use of a radar transponder. Aircraft position is located by radar 
"echo" response and altitude by a "reporting altimeter" reading being returned to 
the radar system encoded in the transporter return. Use of a satellite based LAN 
or WAN will provide an "intranet in the sky", providing much more accurate GPS 
position, altitude, heading, speed and other navigational information to the FAA 
and other operators and computer tracking and monitoring stations, thus enhancing 
collision avoidance information. 

BSPR' 

In its preferred form, a plurality of sensor units, which may include at least 
one video or image sensor/device and/or at least one audio sensor and/or at least 
one motion sensol, are placed strategically about the interior exterior of 

the commercial transport and at strategic ground based locations. In addition 
strategically placed motion detectors, fire sensors, smoke sensors, door or latch 
sensors and other monitoring equipment are incorporated in the system A 
comprehensive system incorporating these various sensing devices Provide a broad 
based, multimedia safety, security and surveillance system for monitoring 
commercial transports at any time, whether or not attended. 

BSPR* J 
In addition to safety and/or surveillance issues, the comprehensive data 
Sl^ection scheme of\he subject invention provides a system permitting enhanced 
monitoring and/or response to crew generated work orders or re -supply orders, and 
ma? even avoid the requirement that the crew order certain supplies. For example, 
SJ^monltorlng the fuel, fresh water, waste water and/or hydraulic levels onboard 
and transmitting this to a ground station, refueling, water delivery and/or 
hydraulic fluid check and supply may be initiated by the station crew and 
crenared for delivery when the commercial transport arrives in port. The 
performance parameters of the commercial transport may also be monitored and may 
be utilized for initiating maintenance procedures, for example, even before the 
commercial transport is in port. Pre-f light or pre-mission checklists may be 
enLnced or automated by monitoring the critical functions and criteria via the 
svstem of the subject invention. The system of the subject invention greatly 
enhances maintenance procedures and efficiency. Where desired, the system is 
capable of permitting'^the commercial transport to carry its detailed -"-^ntenance 
record onboard, permitting full access to such information at remote locations, 
^hrm^intenanc; Record can be routinely updated or P°J^^f, ^.^^/^^l^f ^^i^^^J^, 
maintenance station using the system's unique uplink capability _ The ^^^l^^y to 
both send and receive information will support remote control of the commercial 
transport onboard systems such as lighting, strobes, alarm setting/resetting 
environmental controls, locking systems, siren or other audible signals, fuel 
flow, fire detection and the like. 



?he system also permits full situational awareness capability where all ground or 
water^ransports in the are provided with GPS l^-^^ion information such that th^^ 
ground crew will know where all assets are at any point in time. This can provide 
both collision avoidance as described and also check to assure that the 



both collision avoiaaiiut; ^^o^j- — 1. 1 ^„ 

transDorts are in an authorized area. A composite of all transport location 

reconstruction of events, such as a security breach or collision, is required. 

?hrcomnrehensive multi-media system of the subject invention permits the 
coll^ctfof^nd dissemination of "^virtually all, data — -^^^J J^^^^jj^.^r'^ 
transport at any time, both while in port or m service In the Preferred 
embodiment a comblHitlon of sensors systems are used, with ."^^^^f ^ . 

installed within the transport, on its exterior and at ground-based locations for 



3 of 15 



9/11/01 8:59 AM 



'. , r: http//westbrs:8820/bin/gate.exe?f=doc&s...c_2=&p_doc_3=&p_doc_4=&p_doc_5=&p_doc_6= 
Record Display Form ^ 

the subject invention can communicate via standard ground to air ra ^ 

anywhere in the world using satellite communications. 

at ground bas.d .its. where comm.rci.l transport is located in ora.r 
'ovLents around the perimeter of the monitored """""'l "'^^^"ivrpin, t??t 
deeirabl. to inolud. foouaing Si!!i?f 4°.^;r„:j';ri«lv.t;d 

and/or Joom (x,y,z) po.itioning o.n I^J f """^"dV a GPS sy.tem 

"?^"af to"tri»l«?^rooS:"clS rr:n^p"t"Ly"'t?StSS?iy%.n an or 

o r f L°fS^flVon: Sr ^t^t.£^^^^ SLh'f 

iiElB:^t:»«i'rr?e-^?^^ 

KJ^ral video oa„er.. may he plaoed euoH that the len^^^^^ 

window opening provided in the „d other functional components 

KSSr^rtir-rou-nd fr'-i iC^JSSSLSf tK^J-^ir-oIaJ^tranapor? may alao he added. 

. -4= *-^^r. fr-oTTi the Dluralitv of sensors on the transport 

Siiiuj-^o^S? SSi"^?^ra o 1 K ^^^oo^^^^^ 

using ground P°---.°-^^°^™"^fSsSted for'^any reason such as, by way of 
will operate even if power is disruptea ^or si y simplest form, 

example, tampering by unauthorized personnel or by ^^^^-^J^.^V^/^^^the site 
only'triggered (-tivated) -"-^ , ^^.r^hrUns^r', ^nd on^J the signals 
rnrr:.:d^S'^Ir:^rraLmi?t:f trt e se^^^^^^ 

Ta^^SlS? uLfuf iraraid^Jn^recL^fructin^^: ^^ents in a ..poiEi^enf. 



investigation . 



BSPR: , ^v,^ Hni-erface into the LAN. The LAN transceiver can accept 

The LAN transceiver is the ^"^erface in^o ^n enable the multi-media sensor 

software downloads from various system ^l^-^^^^^^^^^^^Ji^t ions . Other sensors may 
system to be maintained or upgraded ^° P^^f ^^.^^J^^^^^^rs, smoke and/or fire 
also be incorporated - t^^^ ^J^^^; coSigur^d for selecti;ely transmitting all 
sensors and the like, ane system xt=. , basis i e the data is 

of the data on a "-f ^ ;-:^3l^^'^^3Uch a^' com;r;ssion/decompression, 

delivered with °nly delays for processing ^^^.^^ :nonitors 

multiplexing and the l^l^e. The ^Y^^em i P cameras, in 

access to serial, ^y"=h^°"^^^^J^i}/to provide split screen or multi-monitor 
sequential viewing, or alternatively to provia« f 



9/11/01 8:59 AM 



Record Display Form 



http://westbrs:8820^in/gate.exe?f=doc&s...c_2=&p_doc_3=&p_doc_4=&p_doc_5=&p_doc_6= 



viewing of a plurality of cameras. The system may be hardwired or wireless 

transmission may be utilized to further minimize the possibility of a malfunction 

at the onset of a catastrophic occurrence and to make the system more tamper 
resistant . 



BSPR: 

The comprehensive surveillance/communication of the subject invention supports 
communication of monitored data and/or commands or operational data between the 
ground or base station and the transport, between the transport and ground or 
terminal support vehicles and/or equipment, between the transport and various 
monitoring stations or systems, between transports, between the ground station 
and the support vehicles, between the monitoring station and support vehicles and 
between the monitoring stations or systems and the support vehicles . This permits 
the ground station to monitor and/or determine the identity, location, and 
heading of any vehicle in its range for tracking and collision avoidance, as well 
as monitoring sensor information, alarm conditions, emergency conditions, 
servicing requests, maintenance information, navigational information, requests 
for information such as flight plans, weather information, route maps, message 
traffic such as e-mail and the like. Similar information may be transmitted and 
received between transports, between transports and support vehicles and any of 
these and the ground station. The ground station may also send operational 
commands to the various monitoring systems both on-board the transport and ground 
mounted, such as camera tilt, pan and zoom and sensor activation. Other command 
signals such as "lock-on" a specific condition or transport, sensor download, 
activation such as "lights-on" or alarm (e.g., siren) activation and the like. 



BSPR: 

In a typical application, when an alarm from a specific transport is sent to the 
ground station it will be tagged with the GPS coordinates of the transport. The 
alarm will also be reported to a security system, typically including a 
computerized center that distributes the information of the wireless LAN and. 
where used, the wired LAN. The mobile and/or personal security units will also 
report their GPS coordinates to the central computer so that the system knows the 
location of all security personnel at ant point in time . Once the alarm signal is 
received, the system can search and identify the closest appropriate personnel 
and alert them of the alarm condition. This is accomplished by calculating the 
length of the vectors between the transport GPS and the various personnel GPS 
signals. The shortest vectors are the nearest personnel and these can be alerted 
to respond to the alarm condition. 

BSPR: . 

It is a primary object of the subject invention to provide the means and 
apparatus for a comprehensive, multi-media traffic management, surveillance and 
monitoring system for monitoring and tracking a commercial transport vehicle 
while in port or while in route. 

DRPR : 

FIG. 7 is an illustration of an aircraft as an exemplary commercial transport and 
shows the incorporation of on board systems with the comprehensive tracking and 
monitoring system of the subject invention. 



DRPR: ... . 

FIGS. 9 is an expansion of the system shown in FIG. 1, utilizing a remote 
receiver and monitor station in combination with hardwired ground components, 
wireless ground components and an aircraft system interface. 



DEPR * 

It will be readily understood that the various components and features of the 
subject invention can be utilized in connection with a tracking, security and/or 
surveillance system for any of a variety of commercial transports. For purposes 
of brevity, the features of the invention are described in detail herein as 
applied to commercial aircraft. This is primarily because it is assumed that 
aircraft systems are likely to incorporate the most complex and comprehensive 
surveillance systems of the subject invention due to the importance of securing 
this commercial transport while on the ground and both the importance and 
complexity of monitoring and tracking same while in port or in route. The system 
may be scaled up or scaled down depending upon application. For example, land 
vehicles such as railroad rolling stock or over the road trucks may need only 
door sensors, motion sensors and brake monitors, whereas aircraft, as described. 
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will require a substantially more comprehensive system in order to provide 
adequate surveillance. In the preferred embodiment for aircraft as described in 
detail herein, the comprehensive surveillance system utilizes the onboard 
aircraft system in combination with a ground-based wireless system. The wireless 
configur ation can also be applied to the sensors on board the aircraft using the 
same architecture as described here for the ground based portion of the system. 
That is, the on board elements may be hardwired, may communicate through wireless 
radio or may utilize wireless LAN as herein described, or a combination. The LAN 
radio' provides a wireless LAN connection to other system elements. This is a 
well-know but evolving technology that allows high bandwidth wireless data 
transmission between multiple devices. Several different techniques are available 
from a variety of manufacturers, including Raytheon Systems Corporation, the 
assignee of the subject invention. Many of these techniques may be utilized in 
the subject invention. 

DEPR * 

The comprehensive system includes various condition sensors, motion and audio 
detectors, video cameras, light detectors, sound detectors contact switches, 
temperature detectors and control systems for controlling light, and sound 
transmissions to the aircraft. A temperature and/or humidity detector may be used 
for general monitoring functions such as predicting the icing °f the wings in 
winter conditions, or for fire alarm functions. The temperature detector may be 
an? known form for temperature transducer, such as a PTC, NTC, thermistor or 
semiconductor element. More advanced semiconductor elements may be used, such as 
integrated circuit types that may include integral temperature and/or humidity 
sensors, references, analog/digital converters, protocol engines and serial 
driver. Further, integrated circuits can incorporate on-board digital radio 
elements such as DSP based radios to be completely integrated self-contained 
chips ?he temperature analog/digital converter adapts the ambient ^-^P^-^ure of 
the environment into a digital data stream. This digitizer runs at suitable rates 
for continuous temperaturl monitoring. A signal processor can be ^-^^ to provide 
correction to the temperature and/or humidity elements, such as processing out 
non-linear characteristics of the sensors. It can also be used to look for 
profiles such as rapidly rising temperature/humidity _ conditions that may indicate 
a ?ire or open door or other security breach. Detection of such an event would 
trigger a specified unique alarm condition to be transmitted back to other 
elements of the system. 

llT'of the most significant factors in determining the overall complexity of the 
Sstem is the cost associated with the various sensor components. For example, m 
certain applications it may be desirable to add a humidity detector or a carbon 
ZnoTa. llleator. A digitL camera may be used or an ^^^.^^l^^^'^ 
in combination with an analog to digital converter, or digital "^^^ internal 
diaitization circuits, or digital compressed with an internal analog to digital 
converter and a motioA video compressor. In the preferred embodiment, the camera 
runlatful? motion rates. However, it will be readily understood that the camera 
can run at lesser rates for still frame or step video applications. In all cases, 
accurate information can be supplied on a "real -time" basis, i.e., the 
information can be transmitted, received and acted upon by man or machine in a 
timely fashion, sometimes with slight delays, to permit adequate f^^P°"^^^^° 

ke vid;o analog/digital converter is ^--f 5° f /;^,^^:^°?his^ 

::^^r^T^^^^^f^^o^^^^^ - r-or cou^ 
operate at lesser rates foF-itTll frame or step video applications The signal 

motion detection, but the most general method involves capturing repeated video 

imaae and the changes in such characteristics, may also be used. The 
iroce;sor/compresso? subsystem can also be used to image process the video for 
purposes of contrast enhancement, dynamic range improvement, noise reduction 
^nH/orother well-known video processing methods, or other circuitry so 
con^iSurSS to perform the processing by well-known techniques. When the video 
processor/compressor is used for motion detection, any detection will generate a 
specified unique "alarm condition" to be transmitted to other elements of the 
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system . 
DEPR: 

FIG. 1 is an illustration of a basic ground based security and surveillance 
system for aircraft . The aircraft 10, 10a, 10b. . . lOn will be within the view 
of video sensors or cameras 210, 210a . . . n when on the airport ramp. The video 
processor/compressor can also be used to perform still image compression to 
reduce the amount of data required to be transmitted over the network. This can 
be accomplished by using any suitable image compression algorithm, such as the 
industry standard JPEG algorithm, wavelet compression, DjVu from AT&T, or other 
techniques. For full motion video surveillance applications, the compressor 406 
may be used to provide bandwidth reduction motion video transmissions. In this 
application, the amount of data representing a full motion video stream would be 
reduced by using full motion video compression techniques such as Motion JPEG 
compression, MPEG compression, motion wavelet compression, or other techniques. 
This allows better bandwidth utilization of the wireless and wired communications 
channel used by the system. 

DEPR: ^ ^ .n . 

The aircraft will transmit various identification signals, such as tail number, 
GPS location and the like, as indicated at 12, 12a . . . n, to a ground based 
receiver 14. The camera 210, 210a . . . n will also transmit video signals to the 
receiver 14, as indicated at 15, 15a ... n. The location of the cameras will be 
fixed, but may be either permanent locations or "drop and place" movable units 
dispatched as needed, based on changing security situations. It is also possible 
that portable cameras will be transported by the aircraft then deployed on the 
ground, permitting ground surveillance in those airports where a permanent ground 
security system is not installed. The GPS coordinates of ground based cameras 
will be stored at the ground or base security station 18, or as preferred in the 
case of drop and place units, will be sensed by on-board GPS receivers and 
transmitted to the base station. The received videos from cameras may be 
converted by optional converter 16 as required and transmitted to the monitor of 
the ground based security station 18. The converter is used to provide 
compatibility between the transport's format and the ground system format. For 
example, for analog transmission an aircraft may transmit analog NTSC video in 
the United States and PAL in England, Digital transmission may be accomplished by 
placing the converters at each camera transmitting unit (see FIG. 3) thereby 
supporting digital data transmission for permitting transmission by the preferred 
wireless digital system, such as a LAN or W-LAN. 

DEPR * 

By monitoring the identification information from each aircraft, the transmitted 
video format from the various cameras can be matched to a specific aircraft. The 
signal is displayed on a monitor at station 18 where it can be viewed and 
monitored for surveillance and security purposes. In the event of a breach of 
security, security personnel may be readily dispatched to the correct aircraft 
using the GPS location signal to define an accurate position of the aircraft. As 
will be described, the security signals generated by the system of the subject 
invention may also be logged and inventoried for later play back which is 
particularly useful for reconstruction of events. It will be readily understood 
that the ground components of the system may be hardwired, or other forms of 
wireless communication, such as, by way of example, a wireless local area network 
(LAN) could be utilized using radio frequency or optical communications methods, 
as will be readily understood by those who are skilled in the art. The system can 
also be modified to transmit signals from the ground-based station 18 to the 
various ground sensors and aircraft sensor systems. For example, a camera_210 can 
receive and respond to remote positioning and zooming signals. Audio warning and 
activation signals may be sent to the camera locations and to the aircraft to 
activate audio commands, sirens, lights and the like, which are integral to the 
system . 

DEPR: . . 

FIGS 2a and 2b show two different schemes permitting transmission of monitor 
system data from a transport 10 to a base station monitor 18 using a wireless 
transmission scheme as indicated at 12. In FIG. 2a, the camera or sensor (for 
example camera 29) produces a signal which is transmitted as generated by the 
aircraft transmitter 76 to the base system receiver 14 and then converted at the 
base system by format converter 4 00 for processing or viewing at the base station 
in its native format. Where desired, the converter may be at the sensor site as 
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indicated in FIG. 2b. Of course, depending on the various systems being utilized, 
multiple conversion steps may be utilized. Format conversion capability is 
required in order to make the system global in nature. For example, the format of 
each aircraft is often dependent on the country of origin. The United States and 
Japan generally use an NTSC camera format. France and Russia use SECAM. The 
United Kingdom typically uses PAL. It is important that the ground or base 
station be able to recognize and convert any of these formats to a suitable 
format for processing by the base. Compatibility with multiple, yet different, 
systems can be automatically accomplished. Instant protocol detection and 
conversion is shown and described in my copending application, Ser. No. 
08/816,399, filed on Mar. 14, 1997, entitled: "Instant Protocol Selection Scheme 
for Electronic Data Transmission via a Distributive Network" . 

DEPR ' 

As shown in FIG. 5, the camera system includes a base or mounting bracket 56 for 




(y ___ 

provided (z direction) . The preferred embodiment of the system also includes an 
audio sensor such as directional microphone 60. The audio sensor may be an 
acoustic transducer, such as a microphone, that collects audio information from 
the surrounding area. The collected audio can be processed to detect potential 
emergency conditions such as a gunshot or an explosion, or can be routed directly 
back to the monitoring station. Using the sensors of the subject invention, 
locational origin of an explosion or a gunshot or the like can be triangulated 
from multiple sensors and the positional origin can be calculated and displayed 
on maps as an overlay for assisting in pursuit of a perpetrator. The calculated 
origin can also be correlated by computer to the nearest appropriate emergency 
assets, base upon their known positions, and those assets may be automatically 
dispatched. The audio analog/digital converter adapts the acoustic signal 
representing the audio environment into a digital data stream. The digitizer runs 
at real -time rates for real -time audio monitoring. The audio signal 
processor/compressor has two functions. It is programmed to perform detection m 
a number of different manners. For example, the processor algorithms can be 
adjusted to detect impulse noises such as gunshot or a small explosion. Detection 
of such an event would trigger a specified unique "alarm" for that condition to 
be transmitted back to other elements of the system. Other types of detection are 
also possible. By using frequency analysis transforms and signature profiles, 
noises from engines, door openings or other distinctive noises could be detected 
when warranted by the situation or condition. For audio surveillance 
applications, the compressor can also be used to provide bandwidth reduction for 
audio transmission. In this application, the amount of data representing _ a 
real-time audio stream would be reduced by using audio compression techniques 
such as LPC-10, or other well-known or proprietary algorithms. This allows better 
bandwidth utilization of the wireless and wired communications channels used by 
the system. 

DE PR * 

An integrated GPS receiver 64 is provided for generating location information. 
This is particularly useful for "drop-and-place" sensors as opposed to permanent 
sensors. Other features such as a laser range finder 66 that can measure distance 
to objects/personnel may be incorporated to further expand and enhance the 
capability of each sensor component. The camera system shown has full 360 degree 
field of view capability which may be controlled manually by remote control 
signals, may be programmed to pan the area on a time sequence, may track a moving 
transport using GPS signals from the transport or by using image processing 
"tracking software" processing the camera image, or may be responsive to and 
activated by an event occurrence such as from sensors distributed throughout the 
ramp areas, reporting activity over the LAN, in the well known manner. The range 
finder 66 permits the tracking system to locate objects in a precise manner and 
then provide control signals to permit accurate surveillance and monitoring o£ 
same, such as zooming the camera or positioning of other sensor elements. An 
onboard dual GPS systems on the aircraft, with one GPS at the tail and one at the 
nose, used in conjunction with the GPS system 64 permits the system to determine 
size heading and distance to the aircraft being monitored, providing accurate 
location information and permitting the camera to automatically adjust to monitor 
the entire aircraft within its range. This permits the selection of the correct 
camera when multiple cameras are available and permits a wide range of viewing 
possibilities by being able to determine what portion, if not all, of the 



8 of 15 



9/11/01 8:59 AM 



Record Display Form http://westbrs:8820Wgate.exe?f=doc&s...c_2=&p_doc_3=&p_doc_4=&p_doc_5=&p_doc_6= 

aircraft is to be monitored at any given time. In those instances where the 
aircraft is equipped with a single GPS system, much of this versatility is 
preserve d. However, it will be understood that aircraft size then would have to 
be determined from the aircraft type or by optical means. When the transport is 
not equipped with the GPS system, the other sensors such as the range 
finder/ti-acking camera or ground level sensors would provide data for cameral 
selection and updating of electronic situational maps. Each sensor and/or camera 
may incorporate a motion sensor and/or an audio sensor activation device so that 
the system may be activated when a sound or a motion occurs within the censor 
range^ The motion detector may comprise any transducer unit that detect the 

presence of an intruder and can be a device such as an infrared motion detector 
a thermal sensor, an ultrasonic detector, a microwave detector, or any hybrid of 
two or more of these detectors "fused" together to gain better sensitivity and/or 
improved detection accuracy. A motion detector convertor may be ^"^^^P^^^^'^^J" 
convert the signal from either a single motion detector sensor or a battery of 
sensors to digital form for processing and/or transmission to other system 
elements. Multiple elements may be contained within a ^^^^^^^^^^^^^ , 

nackaae or may be fused for multiple sensors in geographically distributed 
element; with data to be fused being transmitted over the LAN. The motion 
detector signal processor is adapted for analyzing the sensor data streams from 
one or more sensors to provide for better sensitivity or improved detection 
accuracy. Well-known techniques may be implemented to process the transducer data 
and detect surges over the set thresholds that represent detection. The 
procefsScompressor can also be configured to accept input from --^^iple sensors 
and process the inputs in a "fused" manner. For example, signals form an infrared 
detector and ultrasonic detector may be "added" together, then threshold 
detec??on performed. This ensures that both an optical and an acoustic return are 
detected before an alarm condition is broadcast. These and other more 
sophisticated well known techniques can be used together to gain better 
sensitivity and/or improved detection accuracy. Detection of such an even would 
trigger a specified, unique alarm condition to be transmitted back to the other 
elements of the system. 



?^ically, the sensors will "sense" the presence of unauthorized activity and 
a^ivate recording from the various audio and/or video equipment and ^^^ivate 
alarms This will initiate the generation of a signal at each of the activated 
units The generated signals will then be transmitted to the monitoring and 
recording equipment, as described, to permit both real-time surveillance and 
recordation of activity at the site. Motion detection may also be determined 
using video time/change techniques in the well-known manner. 



??G 6 is an expansion and further refinement of the system of FIG. 1 and is a 
diJqrammatic iUustration of the system of the subject invention as configured 
forTwTreiess local area network (LAN) . In the preferred embodiment the aircraft 
lo willinclude a comprehensive in-flight security system, as better shown in 
FIG 7 which is cutaway diagram of atypical commercial airline fuselage 10- with 
the 'ca^qo hold 12, the passenger cabins 15, 16 and the flight deck or c°^kpit 21 
partially visible and a plurality of sensors 19a-n. A more detailed description 
^f this onboard system is shown and described f "^^^8/745 ' S^f " In 

c; 798 458 and copending applications Ser. Nos . : 08/729,139, and = ' ^-^^ • -^^ 

the subject invention, \hf currently available sensors may be utilized, without 
additional enhancements or a number of additional sensors may ^e added^ For 
example ground surveillance could be accomplished using only the on-board 
sensors' on ?he aircraft. In the example, a number of video image sensor devices 
sensors on exam ple analoq video cameras, may be mounted inside the skin 

^fth^ krcr^ft ^nd aimei ;hrough%enings or windows provided in the fuselage to 
focus on critic al components of\he aircraft, such as the landing gear cameras 
20 22 the wing engine camera 24 and the tail camera 26 Similar devices or 
ca;e?i; may alsl be'strategically Pl^fd throughout the -terior of the a..c.aft, 
such as the passenger cabin cameras 28, 30, 32, 34, 36, 38, 40, cne cargo ^'*y 
cameras 42? 44, 50 and 52, and the flight deck camera 46. The sensors 19a-n may 
include smike ^nd fire detectors, motion detectors and audio sensors 
strateJicSy placed throughout the aircraft, both internal and external of the 
fuseJaqe The placement and number of devices is a matter of choice depending 
upon the configuration of the ...... ft and the level of surveillance desired. In 

the preferred embodiment the on-board aircraft sensor system is used in 
combination with the ground based system to provide a comprehensive surveillance 
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and security system of the aircraft while on the ground. 

M?^^'=r,or.Hfir^ -reference to FIG. 6, in the preferred embodiment the aircraft 10 
llll S^rincludf a noL GPS sensir 200 an5 a tail GPS sensor 202. The dual GPS 
sensors permit redundancy, very accurate location and directional posxtionxng of 
the qrounded aiLraft^ as well as providing information identifying the size of 
Sc?a?t An aircraft reference signal (such as tail number) country of o^^S?;"' 
aircraic /^n incorporated in the transmitted signal so that the 

^^nitiring s^:tion ;an Id5nti?y tL aircraft, its location and the --rity 
condition thereof by monitoring the signal from that specific aircraft . In the 
wireless embodiment shown, the aircraft is equipped with a wireless transceiver 
204 for transmitting all of the collected signals from the sensors and cameras 
via the wireless network represented by the wireless communication "cloud" 206^ 
^h^ ifrele" ^y^tem shown in FIG. 6 permits transmission ^^^^.^^^^^^^-und 
"^'^^^,.,1 tnwf-T and security, but expands the transmission of data to all locations 
and stitJons which are par; of the'^wireless system. For example, the signals may 
r ,^^n^m?tted to a patrolling ground security vehicle 208, a portable monitoring 
sta I^n 2 8 and/or tf thf groLI security cenLF^TIaThe wireless LAN transceiver 
2^2 ?n Addition, signals may be transmitted in either ^. "^^^^ 
from any unit in the wireless system to any other unit therein. This is 
particularly useful when trying to coordinate a response to an incident in a 
quick response mode. 

?hf wireless LAN 206 or other wireless communication system provides a connection 
ine , „ fixed around resources via transceiver 212, mobile 

arounrresLflfl^h a; l^e security vehicle 208, portable ground resources such 
fs tlt p^r^abirground security statioIT^irind various f^-^-^^J ^r'th^ 
centers such as the control tower 216, the operations control center 220 the 
security center 222, the maintenance center 224, the maintenance hangar 214 and 
the airport fire station 226. 

link to other system elements. The security venicxe zuo is e^uj-j-^P „ 
ii^wing capability as well as an alarm annunciator f ^^^.^^^t 
ouick response Typically, the transmission of an alarm signal by the aircrarn 
3nf trlqge? I iS-up at the various monitoring units and will interrupt 
roiiineif monitored signals. The alarm signal will include aircraft 

^v^t Heal security information while on foot patrol or making rounds, permitting 
identity or based upon their GPS determined location. 

?hf system wireless LAN transceiver 212 operates as the gf^way to the ground 
SLdr^rrmanent wired facilities^ Ltran^t) ^'kfrL^^Ir^is rt^^iSl industry 
re^r^L^i^^e^n'knoin'^trtliosrskili^nr^L S, and may be i^Stalled in many 
type, as is weij. r^n _ t.tv,^v.o 1-*=-^ the svstem may be connected to remote 

nodes as ' ^"""^ . . corporate operations or aircraft manufacturer 

o^Ir^tton:? f:r;x:mpie"%hr?:u:er m^y be conf igu^id^eeded with typical 
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commercial techniques, such as firewalls to protect access, protocol converters, 
and encryption devices, as needed* to direct secure or unsecured information to 
the various ports, nodes and centers. 

DE PR * 

Where desired, only preselected alarm signals may be transmitted to selected 
centers. For example, any heat or smoke detection, fuel spill detection or 
medical emergency would generate an alarm signal at the fire control center 226. 
The maintenance hangar may have access to fluid sensor data and stored 
maintenance requests and records. Thus the system can be configured m an 
information hierarchy format where only useful information is forwarded to the 
various centers . 

DEPR ' 

The use of the dual GPS receivers 200, 202 on the aircraft 10 permits the 
reporting of the general location of the aircraft on the ramp during taxi when 
parked whether or not attended. The use of two GPS receivers provides redundancy, 
better accuracy and orientation information for the aircraft by reporting two 
distinct position datum signals. It will be readily understood by those skilled 
in the art that other position signal devices could be utilized such as, by way 
of example a single GPS receiver and a magnetic compass (which may have to be 
corrected for local magnetic fields or interference). By linking the position and 
orientation information to the ground based centers the location and orientation 
^ of the air craft at all times it is on the ground the aircraft may be closely 

monitored. Such a system provides ground control transmitting signals showing the 
location and movement of all aircraft while on the ground, in much the same 
manner the radar transponders provide air controllers with position and movement 
data while the aircraft is airborne. This is particularly desirable when the 
movement of aircraft is portrayed on a map display. Other ground vehicles such as 
fuel trucks, waste water trucks, baggage handling trains, security vehicles and 
the like can also be tagged with GPS receivers and LAN transceivers for 
monitoring their position relative to the aircraft on the ramp. An automated 
computer system can be operating in the background looking for potential 
collisions and generating alarm messages if such a conditions is detected. 
Another automated computer function can track vehicles relating to their 
authorized areas and issue alarms if security is breached. Yet another function 
can track the presence or absence of needed services, such as the timely 
appearance of catering trucks, fuel trucks, wastewater trucks, baggage trains and 
the like after the arrival of a subject transport. If any of these required 
services do not arrive at the transport within a prescribed time period, and 
"alarm" can be reported over the LAN to the missing services vehicle, and/or to 
the responsible operations center. This function can be completely automated by a 
controlling computer system. 




throughout the airport. When an an . ■, , j ^^=^-k-= 

airport ramp 314, the various cameras 210a-n and or other ground based sensors 
will provide a secure area for the aircraft. Any activity within the range of th. 

cameras may be viewed and monitored. 

DEPR * J 
The system of the subject invention is designed such that aircraft onboard 
sensors and ground-based sensors may be used in combination to provide a 
comprehensive security system. The ground-based sensors may "^^^^^^^^^^^^i^^ 
provide basic ground security. The aircraft sensors may be used alone to Provide 
some ground baled security with a minimum of modification to existing hardware. 

DEPR * 

In the embodiments shown and described, a multi-media recorder is utilized to 
record the information for archival purposes. This can be a ground based recorder 
or the aircraft "black box" recorder 58 (shown as installed in the tail section 
of the aircraft, see FIG. 7) may be utilized, in the same manner as the current 
data and voice black boxes (not shown) . 
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DEPR: . ^ ^. 

Audio and video monitors are also provided at the base security station to 
provide near real -time surveillance. The flight deck monitor and control panel 54 
is located on the~H3Ktrol panel in the cockpit 21 will also have access to this 
information. Other monitors may be provided where desired. 

DEPR * 

Turning now to FIG. 9, the system shown is adapted for wireless installation 
using both onboard aircraft sensors and ground based remote sensors. The system 
shown relies on the standard on-board radio of aircraft 10 to transfer all 
aircraft signals to the base station receiver 81 via antenna 81a. In the 
alternative embodiment of FIG. 9, the ground-based cameras (camera 210d) and a 
motion sensor 31 are hardwired as shown at 87 to a controller 85. The on-board 
signals are input from the receiver via hard wire 89. Wireless ground based 
sensors such as the camera 210a and the motion detector 31a may be used in 
combination with the hardwired ground based cameras (see camera 210d) or other 
hardwired sensors. This permits maximum flexibility of the system architecture. 
The wireless signals will be transmitted via a dedicated sensor array 
transmitter/receiver 83 and antenna 83a. 

DEPR * • 
As shown in FIG. 10, the use of a wireless network provides maximum versatility 
in the transmission of information and the monitoring and processing capability 
provided by the system. As indicated in FIG. 10, the transport 10 both sends and 
receives information between the ground station 18, as previously described and 
as indicated by the wireless data path A. The transport may also _ transmit and 
receive between the fixed sensor station (s) 20 as indicated by wireless data path 
C The fixed sensor station is also in direct communication with the ground 
station as indicated by wireless data path D. It should be understood that 
permanent installations such as the ground station and the fixed sensor station 
could be hardwired with one another without departing from the scope and spirit 
of the invention. In addition, support vehicles such as, by way of example, the 
baqgage train 13 may be equipped with sensors such as location sensors and the 
Tail generated by this sensor may be transmitted to the ground station via path 
B the monitor station via path E and directly to the transport via path F. The 
around station 18, monitor station 20 and transport 10 may also communicate 
directly with the ground support vehicle 13. For example, if the ground support 
vehicle comes within a designated "keep-out" or no trespassing zone or is too 
close t o the transport, a proximity sensor or calculated from the GPS data may be 
utilized to activate and send a warning signal to the ground support vehicle. As 
indicated by wireless path G, sensor data may also be communicated between 
multiple transports 10 and 1.0a. 

?hf comprehensive system of the subject invention not only provides surveillance 
of the aircraft while at the gate or while unattended, but also provides taxi 
pro^ectflffHl^monitoring. As^hown in FIG. 11, when all ground vehicle -ch as 
fuel truck 11 and baggage train 13 are outfitted with GPS receivers as well as 
the aircraft 10, the loLtion and safe distance of each vehicle and the aircraft 
may be monit ored. "Train" type vehicles may be outfitted with two or more GPS 
receivers to relay the length of the vehicle . Each car can have a separate 
moduir A computerized map of the airport tarmac T, the taxiways P and runway R 
can be generated showing the position, direction and movement of each vehicle and 
the aLcraft . Predefined keep-out "zones "Z" may be established and an alarm may 
be sounded i f the zones are breached. Also, prescribed areas ^ °^/^^horized 
vehicles may be established and monitored. If a vehicle is outside the designated 
area or breaches a zone "Z", an alarm condition will result. This can be 
prioktLed as a cautionary breach, a dangerous breach and so on, depending on 
proximity of the ...vno„. vehicles and aircraft to one another. For example if an 
aircraft 10 comes too close to a fuel truck 11, alarms in the aircraft, the fuel 
truck wi ll be activated. In the situation advances to a danger zone, a second 
ilarm condition may alert ground or base personnel that a breach has occurred so 
the intervention may be initialized. Logging of the "safety" breaches can be made 
so that safety improvements or training may be implemented based on need. 

DEPR * 

A combination of ground sensors in a matrix on the airport ramp (see sensors 
210a 210n in FIG. 8) will scan and monitor vehicles . If a vehicle is detected 
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that does not have a GPS identification authorized for that location and alarm 
condition will result. For example, if a stray baggage train 13 entered the 
taxiway area, an alarm would sound indicating that the train 13 has entered an 
unauthorized area. Emergency and security personnel may also be alerted and 
dispatched if unauthorized or untagged (no GPS identifier) vehicles are present. 
This protection scheme could be expanded to include personnel as well as 
vehicles. For example, the ground vehicle can have a sensor that reads a 
personnel security token or device such as an encoded digital key. This key 
information would enable the vehicle and would also be encoded with GPS 
information and vehicle identification, which is transmitted over the LAN. 
Security software can then check to determine if the individual is authorized to 
be present in the vehicle at that time and location, activating an alarm if 
proper authorization is not confirmed. The vehicle could also be immediately shut 
down. Visual identification of personnel may also be accomplished using the 
sensor systems of the subject invention. 

DEPR : 

As shown, a variety of image sensor devices may be incorporated, including the 
video cameras CI, C2, C3 . . . Cn, an advanced imaging device such as the FLIR 
camera 220, the on board radar 222 and the like. All of these produce _ a visual 
signal. In addition, various audio signals may be incorporated utilizing a 
variety of audio sensor devices, such as a cockpit voice sensor 113, on board 
radios 224, 226 and the aircraft public address system 228. All of these produce 
an audio signal. The operational data signals are also incorporated, as 
previously described, and may include the GPS sensor 72, other navigational 
sensors 230, the various intrusion sensors 115 and other sensors 125. Thus, the 
system of the subject invention will accommodate a multiple input, multi-media 
array incorporating video, audio and digital data signals into a comprehensive 
database for providing detailed information relating to the aircraft condition at 
any time. 

DEPR ■ 

The visual and textual data is available at a display monitor 54. The audio 
signal is output at 237 to an audio output system such as amplified speaker 240. 
All of the data, including all video, audio and digital data will be recorded on 
the recorder system 70. Information representing audio, video, sensor data, and 
other vital digital data is fed from the multimedia multiplexer to the recorder 
70 over the signal lines 233. It should be noted that the multimedia multiplexer 
may be analog, digital, or packetized digital data type, or a combination of 
technologies based on application. Where desired, selected portions of the 
systems data on the aircraft mky be downlinked to the ground or base station 18 
(see FIG. 2) as the combined, comprehensive output signal on line 246 to be 
transmitted to the ground station via the aircraft radio system 80 and the 
antenna 82. As previously described, the information may also be transmitted to a 
wireless satellite via transceiver 280 and dedicated antenna 282. Once the 
information is generated as a useable data signal, as indicated at line 231, 233 
and 235 the controller, in combination with commands from ground security, 
controls the collection, monitoring and review of the information. This permits 
access to any single sensor signal, or any combination via line 231 by sending a 
command via line 248 to the controller 241 for controlling the monitor related 
multiplexing switches via line 243 to control the signal output on line 231. For 
example, this may be a single camera view or an array of intrusive motion sensors 



115 . 
DEPR 



vehicle 



DEPR * 

Where desired, a light level detector may be is used for detecting light 
conditions such as the ambient lighting or transient conditions such as ■-— -- 
headliqhts or a flashlight. The light detector analog/digital converter adapts 
the ambient light levels into a digital data stream, this digitizer runs at 
rear-time rates for teal -time illumination monitoring. The light detector signal 
procii^ can be programSiiTTo look for profiles such as rapidly increasing light 
conditions that may indicate a vehicle or a flashlight as opposed to the rising 
or setting sun. Detection of such and event would trigger a specified unique 
alarm condition to be transmitted back to other elements of the system. 



DEPR: 



FIG 15 is a diagrammatic illustration of the placement of tracking sensors on 
the 'ramp and taxiways of an airport for tracking the movement of the commercial 
transports such as transports 10a and 10b as they come into the gate area 350. 
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The sensors S1-S32; are strategically place to track the transport as it proceeds 
along the runway, the taxiway and the ramp. This is particularly useful for 
aircraft which do not have GPS signal generating sensors, making it possible to 
track and identify the transport at any time. Various sensing devices can be 
utilized in this configuration such as acoustic sensors, acoustic return "sonar", 
optical, optical return, microwave, microwave return, contact or weight 
detection, electronic proximity (underground wire), or similar sensors. The 
sensor system detects the transport, and where return sensors are used, will also 
identify the distance. By using sequential sensors, the speed and direction of 
travel may also be calculated. This type of sensor system will also detect the 
presence of other assets or personnel in the area. 

DEPR: , , . ^ . 

FIG. 16 is an expanded illustration demonstrating the calculation and signaling 
of appropriate personnel and equipment to the site of an event requiring 
emergency response. By way of example, assume the tracking camera 210a and 210b 
provided a visual signal indicating smoke at transport 10. At the same time, the 
on-board fire and smoke detectors would transmit a signal to the ground based 
transceiver 212 via the wireless LAN. In addition, the precise location of the 
transport will be known because of the location signal generated by the transport 
GPS sensor 200 which is also transmitted over the LAN. The receipt of these 
various signal will activate several actions. First, all of this information will 
be transmitted to the ground control tower 216 and to the operations control 
center 220, as has been previously described herein. The airport fire station 226 
will be alerted to the indication of a fire and smoke event and the security 
center 222 and maintenance center 224 will receive appropriate information. The 
automated dispatch computer center 225 will monitor the location signal provided 
by the transport, as well as the location signal of on ground personnel 
218a-218c, response vehicles 208a-208c and fire support vehicles 352a-c. By 
monitoring the type of event that has occurred and both the type and location of 
available personnel and equipment, the dispatch center can alert and initiate the 
most efficient appropriate response. The location signals provide sufficient 
information for the computer system 225 to determine by well-known methods, which 
asset is closest. For example, ground personnel 218b is closest and would receive 
the first response signal. If a response vehicle was programmed to respond, 
v ehicle 208a would be first alerted. Likewise, the closest fire truck is truck 
352^ i;;hich would be the first alerted. As back-up is needed, each of the ground 
support assets have the capability of signaling for additional support directly 
back to the dispatch computer. The computer can then select the next closest^ 
appropriate asset. The system of the present invention provides a comprehensive, 
efficient method of collecting, distributing and reacting to critical information 
to maximize the response of appropriate functional vehicles and personnel on a 
real time basis while assuring that assignments are prioritized as set by 
operational personnel. This greatly increases both the timing and the 
effectiveness of response to critical events. 

DEPR: ^ , ^ • 

The multi-media security and surveillance system of the subn ect _ invention 




vehicle is not in use and is left unattended. In addition, the permanent tape 
record will prove invaluable for investigating unauthorized activity or accidents 
after they have occurred. The preferred embodiment of the system is specifically 
designed for new commercial aircraft but is equally well suited for retrofit 
applications and for other safety applications as well, and may be scaled up or 
scaled down depending on application, 

DE PR * 

The video recorders, synchronizing networks and multiplexing and split screen 
hardware are well known and their adaptation will be readily apparent to those of 
ordinary skill in the art. Any suitable video recording format can be used, for 
example, an analog video tape recorder, a digitizer and tape, hard drive or 
optical drive configuration. Digital cameras could be incorporated m lieu of the 
standard analog type cameras currently in use in most applications As digital 
technology becomes more readily available and more cost effective, it is 
contemplated that most of the imaging, monitoring and-recording equipment will be 
of a digital format because of the increased reliability and the minimized space 
requirements. Of course, it should also be understood that the monitoring. 
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transmitting and storage capabilities of the invention are also well suited for 
capturing any video or visual image generated by the on board avionics of the 
aircraft . 
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BSPR: 

Along with recent tendency to process computer-related data in multimedia, 
multimedia has also rapidly found its way to be utilized in automobiles . Types of 
information handled in automobiles are, for instance, navigation information. 
Vies ( Vehicle Information Control System) information, audio/visual information, 
radio, information from on -vehicle telephone, information from teletext 
broadcasting, information received from vehicle systems and the like. 

BSPR: 

A most serious problem in processing information from multimedia is simultaneous 
occurrence of an enormous amount of data. More specifically, a driver in an 
automobile receives a large amount of information e.g. information from VICS, 
information from multiplexed audio broadcasting and the like, from places where 
the driver does not take any part in. The obvious characteristic of the 
multimedia in automobiles is in that, in a case where a driver receives image 
data by a multimedia system, if the driver is not interested in the image, the 
driver simply needs to look away from a screen showing that image, so that the 
driver is not disturbed while driving the automobile . 



Particularly with respect to regulating audio information output, for instance, 
Japanese Patent Application Laid-Open No. 1-142994, No. 5-203458 and No. 5-332778 
disclose controlling of mainly the volume at the time of outputting audio 
information. However, when volume is controlled, a driver is required to 
concentrate even harder on the contents of the information because low volume is 
difficult to hear. 

BSPR: 

Moreover, according to Japanese Patent Application Laid-Open No, 4-1898, the 
length of guidance provided as audio information is controlled in accordance with 
vehicle speed, to synchronize outputting of the guidance information with passage 
of neighborhood of a destination point, whereby making it possible for a driver 
to associate what is seen with the contents of the guidance. 

BSPR: 

In addition, according to Japanese Patent Application Laid-Open No. 5-118866, a 
degree of driver's familiarity with respect to each road is determined by 
accumulating the number of times of driving the road, and in accordance with the 
degree of familiarity, audio information offering is controlled to be turned 
on/off. Since a driver does not need to audibly receive information on a familiar 
road, such unnecessary audio information is not provided. 

BSPR: 

Even if the length of guidance provided as audio information is controlled in 
accordance with vehicle speed as disclosed in the aforementioned Japanese Patent 
Application Laid-Open No. 4-1898, in a case where a large amount of information 
occur simultaneously, a driver may receive information regarding the destination 
after the driver has already passed the destination. 



In other words, all the above-described conventional techniques are not capable 
of handling a case where there are plural information occurred simultaneously to 
be provided to a driver. Since these conventional techniques did not expect the 



BSPR: 



BSPR: 
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arrival of the multimedia era in automobiles, drivers have to bear with useful 
and useless information mixed together. 

BSPR: 

Still further, in the navigation apparatus according to the aforementioned 
Japanese Patent Application Laid-Open No. 8-254437, either one of the two types 
of audio information (the aforementioned first type and second type of audio 
information) is always outputted whenever there is information to be provided. 
Thus, when both types of audio information occur, one of the types of audio 
information is outputted which may be useless to a driver at the time of output. 

BSPR: 

According to the navigation apparatus as the preferred embodiment of the present 
invention, the priority order is prescribed in accordance with a running state of 
a vehicle. 

BSPR: 

According to the navigation apparatus as the preferred embodiment of the present 
invention, the priority order is changed in accordance with change in the running 
state of a vehicle . Accordingly, the navigation apparatus can adequately provide 
necessary information without excessively providing information. 

BSPR: 

According to the navigation apparatus as the preferred embodiment of the present 
invention, the priority order is prescribed in accordance with running 
environment of a vehicle . 

BSPR: 

According to the navigation apparatus as the preferred embodiment of the present 
invention, the priority order is changed in accordance with change in running 
environment of a vehicle . 

BSPR: 

means for allocating, to inputted information, a rank indicating a type of the 

inputted information, and a priority order indicating a priority degree of the 

inputted information; and 

DRPR: 

FIG'. 1 is an interior view of a vehicle showing the main part of a 
multimedia- type navigation system adopting the present invention; 

DRPR: 

FIG. 2 is a wiring view of a vehicle showing the way the navigation system shown 
in FIG. 1 is connected; 

DRPR: 

FIG. 6 is a table for managing attribute data (priority orders, rank, memory 
capacity and the like) allocated to each of the various data that are used in the 
navigation system of the present embodiment; 

DRPR: 

FIG. 11 is a table showing a setting table where urgent levels LVL of output 
timing is set, which is used in the navigation system according to the first 
embodiment ; 

DRPR: 

FIG. 12 is a table for explaining the state of control operation in accordance 
with values of the urgent level LVL; 

DRPR: 

FIG, 21 is a table where variables "limitation LMT in number of times " are 
defined, which are used in controlling of the navigation system according to the 
first embodiment; 

DRPR: 

FIG. 22 is a flowchart showing a "obtaining number of times " routine in control 
steps of the navigation system according to the first embodiment; 

DRPR: 
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FIG. 24 is an explanatory view showing operation of "obtaining number of times " 
routine in FIG. 22; 

DRPR: 

FIG. 33 is a timing chart explaining operation of the navigation system according 
to the second embodiment; 

DRPR: 

FIG. 34 is a timing chart explaining operation of the navigation system according 
to the second embodiment; 

DRPR: 

FIG. 46 is a flowchart showing a "obtaining number of times " routine in control 
steps of the navigation system according to the third embodiment; 

DRPR: 

FIG. 70 is a timing chart showing operation of the navigation system according to 
the ninth embodiment; 

DRPR: 

FIGS. 71A-71F are graphs for explaining the principle of calculating a time 
interval in the navigation system according to the ninth embodiment; 

DEPR: 

Preferred embodiments of the present invention will be described in detail in 
accordance with the accompanying drawings. As will be apparent from the following 
descriptions, the present invention is applicable not only to automobiles , but 
also to other movable bodies e.g. airplanes and ships. 

DEPR: 

FIG. 1 is a front view seen from a driver's seat in an automobile adopting the 
navigation apparatus of the present embodiment. FIGS. 2 and 3A show the system 
structure indicative of a state where the navigation apparatus of the present 
embodiment is installed in an automobile . 

DEPR: 

Referring to FIG. 2, reference numeral 2 denotes a centralized control unit which 
calculates various data (remaining fuel, mean fuel consumption, mean running 
speed and so on) related to running state of a vehicle and integrally controls 
each of the components described below. Reference numeral 3 denotes a LAN unit 
provided in the vehicle for controlling, e.g. an anti-lock brake system (ABS) , 
sensors (not shown) necessary to control four-wheel drive operation and the like, 
and so-called local area network communication between a driving unit and the 
centralized control unit 2. Reference numeral 4 denotes a RAM card drive which is 
a device for reading/writing information from/to a RAM card which stores 
information related to a driver e.g. seat position, mirror position and the like. 
Reference numeral 5 denotes a data storage drive which is a device for 
reading/writing various data stored in a data storing medium such as an FD 
(floppy disk) , MD (magneto optical disk) , PD (phase- transit ion- type optical disk) 
and the like. Reference numeral 6 denotes an audio guidance speaker which outputs 
audio information from a navigation controller 17 via an audio output interface 
included in the centralized control unit 2. Reference numeral 7 denotes a 
microphone into which voice instruction of an operator is inputted and further 
transmitted to the navigation controller 17 via a voice recognition interface 
(not shown) included in the centralized control unit 2. Reference numeral 8 
denotes a display such as a liquid crystal display and the like where a 
navigation screen, a screen for various input operation and conditions (running 
speed, air conditioning and the like) of the vehicle are displayed. In the front 
surface of the display 8, an input apparatus adopting a capacitance coupling 
method or method utilizing infrared rays is incorporated to enable input 
operation by touching the display panel. Reference numeral 9 denotes a portable 
telephone to which a telephone antenna 13 is connected. Reference numeral 10 
denotes an operation switch which is used to perform input operation to the 
centralized control unit 2 and navigation controller 17. The reference numeral 17 
denotes the navigation controller which searches for an appropriate route to 
reach a destination designated by an operator using the operation switch 10 on 
the basis of position information obtained by a GPS (Global Positioning System) 
antenna 11 and map information stored in a CD-ROM inserted in a CD-ROM changer 
19. Based on the searched information, guidance is provided by displaying the 
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information on the display 8 or outputting audio information from the audio 
guidance speaker 6. The map information of CD-ROM read by the CD-ROM changer 19 
is the basic data of route information which is displayed on the display 8. 

DEPR: 

As shown in FIG. 2, to the navigation controller 17, an FM tuner 16 which 
receives FM multiplex broadcasting compatible with VICS ( Vehicle Information 
Communication System) , and a beacon receiver 18 which receives a beacon signal 
from an electric wave beacon antenna 14 and an optical beacon antenna 15, are 
connected. VICS signals and/or beacon signals obtained by the above devices are 
interpreted as traffic control information and displayed on the display 8, then 
used as a condition (restricted condition) of route search at the time of route 
guidance. In addition, area information related to the area where the vehicle is 
running may be input from the data storage drive 5 to be used as display data. 
Since the technique of detecting the position of a vehicle by GPS is well known, 
description thereof will be omitted. 

DEPR: 

The navigation system described below restricts or prohibits to output 
information (particularly, audio information) which is not necessary by a driver 
or which is massive to be provided to the driver. When there are a plurality of 
audio information occurred simultaneously to be offered to the driver, it is 
necessary to select single audio information, i.e. specific information, from the 
plurality of audio information since only single audio information can be 
provided at once to a driver. According to the navigation system of the present 
embodiment, the single audio information is selected on the basis of "importance" 
of the information which is determined based upon an association between a 
priority order of the information itself and a running condition (running state, 
running environment, driving time zone, driving purpose and the like) which is 
independent from the information. Furthermore, when the foregoing conflicts 
occur, some data are deleted under a. predetermined condition. With respect to 
such data, a message indicating that the data will be deleted is displayed on a 
display; alternatively, the audio information subjected to deletion is, instead 
of being outputted audively, converted to text data and displayed on a head-up 
display shown in FIG, 1. 

DEPR: 

The display 8 in the present system includes a touch panel. FIG. 3B shows 
switches used for requesting a type of information by using the touch panel. As 
shown in FIG. 3B, a driver can access to the information in six different genre, 
e.g. "news," "weather," "sports," "traffic information," " vehicle information," 
"gasoline (GAS)" and the like. FIG. 3B shows the function buttons (switches on 
the touch panel) for accessing to these information. Upon pressing any of the 
switches, information corresponding to the pressed switch is provided to the 
driver . 

DEPR: 

The information genre switches (FIG. 3B) are also used to display genre necessary 
in the current running condition. For instance, "GAS" menu is displayed when the 
centralized control unit 2 receives vehicle information indicating that the 
amount of remaining gasoline is small. Such displaying is performed not because a 
driver has requested, but because the system has made a determination on the 
basis of various data. As another example with reference to FIG. 3B, ''weather" 
menu is displayed when wipers start running because the system determines that 
rain has started, 

DEPR: 

The information subjected to be handled by the present navigation system shown m 
FIGS. 1-3A is provided by media such as, NAVI, VICS ( Vehicle Information Control 
System) , audio equipment, radio, on -vehicle telephone, teletext broadcasting, 
vehicle system and the like. As shown in FIG. 4, information handled in these 
media includes: vehicle information, NAVI information, VICS information, 
information from audio equipment, information from on -vehicle telephone, 
information from teletext broadcasting and so on. Specific examples of the 
information are shown in FIG. 4. 

DEPR: 

The navigation system according to the present embodiment utilizes aforementioned 
vehicle information, NAVI information, VICS information, information from audio 
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equipment, information from on -vehicle telephone, teletext broadcasting 
information and so on, as necessary information for navigating operation. 
However, in the multimedi^a navigation system, there may be a plurality of 
information to be provide^d simultaneously. When a large amount of information are 
provided simultaneously to a driver, the driver may feel uncomfortable, and in 
addition, providing too much information may confuse the driver. Particularly, in 
case of audio information, these problems are apparent. 



DEPR: 

Accordingly, the navigation system according to the present embodiments assign a 
priority order to information if there are a plurality of information to be 
provided simultaneously. Alternatively, the navigation system regulates 
information offering performed by a medium which is considered unnecessary under 
a predetermined condition. By virtue of the above, the present invention provides 
a driver with only necessary information. 



DEPR: 

In general, running conditions (e.g. speed, distance between vehicles, auto 
cruise state and the like) vary momentarily. In the first embodiment , the mode of 
information regulation is changed in accordance with the running conditions. Note 
that information in the multimedia navigation system includes various types, such 
as urgent information (e.g. accident information) which should not be regulated 
because of its high urgency characteristic, and non-urgent information such as 
titles of music programs or the like. The mode of regulating information offer 
should be determined in accordance with the running condition at which the 
information is to be provided. Therefore, in the present embodiment, the mode at 
which information is regulated is changed in accordance with the change in 
running conditions. Since running conditions vary depending on the driver or 
roads on which the driver is driving, the running conditions in the present 
embodiment are classified into seven types as shown in FIG. 5: " vehicle state," 
"running state," "running environment," "driving purpose," " time zone," "driver's 
condition" and "information precision." In other words, running conditions are 
classified into seven types, and maps are set to prescribe modes of information 
regulation in dependence upon each running condition. Meanwhile, reference items 
such as those shown in FIG. 5 are displayed to a driver (on the display 8) so 
that the driver is fully informed of how information offer is regulated upon 
selection of each map. 



DEPR: 

FIG. 6 is a table which defines types of information generated in the system 
according to the first to twelve embodiments and various attributes^ 
predeterminedly given to each type of information. As the types of information, 
the followings are set: "urgent information," " vehicle information," "VICS 
information," "traffic information," "NAVI information," "weather information," 
"news," "sports information," "event information" and "music title." The table in 
FIG. 6 shows definitions in "types KND, " "priority order PRD, " "priority rank 
(default value) RKD, " "priority rank (corrected) RKC, " "capacity (default value) 
CAPD," "capacity (changed value) CAPC, " "remaining capacity REM" and "pointer" 
with respect to each of the various information. 



DEPR: . , . ^ 

More specifically, an identifier KND indicative of the type of the information is 
added to each data. For instance, KND=3 is VICS data. Moreover, the "priority 
order PRD" is set in advance for each data. The value of "priority order PRD" ^ 
indicates priority at the time of offering the data. For instance, an identifier 
KND=3 is assigned to data inputted from VICS medium, and the value of the 
priority order PR is 3 , Lower number indicates higher priority. KND-1 indicative 
of the highest priority is given to "urgent information" such as data informing 
that a vehicle is on fire or accident in a tunnel or the like. The "priority 
order PRD" shown in FIG. 6 indicates default values which can be changed by an 
operator performing predetermined operation. The priority order PRC changed by an 
operator is stored in PRC in the table in FIG. 6. 

DEPR: ^ ^ 

FIG. 9 shows a list of input information and data to be outputted which construct 
a queue table. More specifically, the table shows the list of stored information 
which has been inputted by a certain time t3 . FIG. 9 indicates that data=X 
indicative of vehicle information (KND=2) is inputted at time tl; data=Y 
indicative of traffic information (KND=4) is inputted at time t2 ; and data=Z 
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indicative of urgent information (KND=1) is inputted at time t3 . When next data 
is inputted, the data is stored in a position in the table indicated by the 
pointer r. 

DEPR: 

The table in FIG. 13, showing lower limit values of priority, is referred when 
there is information to be provided as audio information, and defines the 
relationship between the running state STAT of a vehicle at the time of providing 
information and the lower limit value RKLMT which regulates the priority rank RKC 
of the information. 

DEPR: 

For instance, it is assumed herein that input information (X, Y, Z) as shown in 
FIG. 9 is stored in a memory to be provided as audio information. When the 
current vehicle is in the state of abrupt break, the lower limit value RKLMT is 
A. Therefore, information having the priority higher than rank value A is 
recognized as the information to be provided to a driver. Thus, the data X and Y 
whose rank values are B and C respectively are not provided to the driver as 
audio information. 

DEPR: 

Meanwhile, when a vehicle is in the state of backing up, the lower limit value 
RKLMT is C. Therefore, information having the priority higher than rank value C 
is recognized as the information to be provided to a driver. Thus, all the 
information X, Y and Z {all data in FIG. 9) whose rank values are respectively A, 
B and C are provided to the driver as audio information. 

DEPR: 

When regulating in formation offer in accordance with the priority lower limit 
RKLMT (degree of priority order) , if there are a plurality of information having 
a higher rank than the priority lower limit RKLMT, priority order cannot be given 
to these information. Therefore in the first embodiment, for the purpose to 
assign absolute priority orders to a plurality of information where priority 
orders cannot be determined by the priority lower limit RKLMT, univocal 
sequential relationship is given to information offering in accordance with 
priority order PRC which is allocated univocally to the information type KND . 

DEPR: 

Note that in order to save the storage area, unnecessary data may be periodically 
deleted. The unnecessary data is, for instance, data which has been stored for a 
predetermined period of time, traffic information with regard to a point already 
passed, title of songs while a driver is not listening to radio and the like. 

DEPR: 

As set forth above, in the system according to the present embodiment, the data 
storage drive 5 is not occupied by particular information which occurs frequently 
despite its importance, because the storage area is set in accordance with types 
of media, thus types of information. In addition, by allowing a user to designate 
a storage area, a larger storage area can be allocated to information which is 
important to the user. Therefore, it is possible to prevent important information 
from not being stored. Furthermore, on account of the special storage area (free 
area) allocated to urgent information, urgent information can be received any 
time . 

DEPR: 

In step S502 in FIG, 17, an argument x of the subject information is saved in a 
register n. In step S504, running state STAT of the vehicle is determined. Seven 
types of running states STAT of a vehicle are assumed in the present embodiment : 
"abrupt turn," "abrupt break," "back up," "change lane," "turn right/left," 
"abrupt acceleration" and "normal." In step S506, the lower limit value of 
priority RKLMT is read out of the table shown in FIG. 13, "definition table of 
priority lower limit RKLMT" in accordance with the detected running state STAT. 
In step S508, the priority rank RKC and the lower limit value RKLMT of the 
requested information data are compared, and in step S520, only the data 
satisfying RKCn . gtoreq . RKLMT is subjected to consideration for information offer. 
Therefore, data n which satisfies RKCn 
DEPR: 

As described above, according to the present invention, when there are a 
plurality of information to be provided to a driver, appropriate information 
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(RKC qtoreq.RKLMT) is selected in accordance with the running state of a vehicle 
and provided to the driver. As a result, information truly necessary by the 
driver is selected, so that the aforementioned problem of making the driver .feel 
uncomfortable will be solved. 

DE PR * 

In the first embodiment, data is discriminated between data necessary to be 
orovided to a driver and data not necessary to be provided to the driver, in 
accordance with running state as defined in the table in FIG. 13. However if 
there are a plurality of data, having the same value of the priority rank RKC, 
simultaneously exist to be provided to a driver, the priority of the data cannot 
be determined by using only the lower limit value RKLMT. To cope with this 
situation, in the fist embodiment, an urgent level LVL of output timing is 
defined based on the priority rank RKC of the data and the current running state 
STAT, and the data occurred at the same time is mediated based on the level value 
LVL (=1 to 3) . 

FIG 'll shows a table on the logic of generating a level LVL indicative of output 
timing of the data (i.e. urgent level for output), which is determined on the 
blilTof a priority rank of the data (inputted data or data subjected to output) 
and the current running states STAT. The level LVL is used to determine the data 
is to be outputted (information offer) as shown in FIG. 12 . More specifically 
when data is in an output queue, determination is made, utilizing the level LVL, 
on the basis of the priority rank RKC of the data and the running state as to 
whether the data should be outputted or should be forced to wait. Meanwhile, when 
data in the output queue is currently being outputted, determination is made, 
utilizing the level LVL, as to whether to continue or terminate the output 
Operation . 

Fll^\2 shows results of control procedure shown in FIG. 18. More specifically, 
the level LVL is used differently depending on whether or not there is data 
curri^^tT^ being outputted (referred to as "outputting data" for descriptive 
convenience) al shown in FIG. 12. In a case where "input data" occurs while there 
is no "outputting data," the level LVL is used to determine how to process the 
inpu? data"^ ^n a^case ;here thi^Tis "outputting data," the level LVL is used to 
determine whether or not the output operation is to be terminated. 

Shen'there is no outputting data, determination NO is made in step S522 in FIG. 
18 In step S550, the level LVLn of input data n is obtained. If the value of 
LVLn is "1," the processing proceeds to step S554 where the input data is 
outputted More specifically!^ an output queue flag OUTQn of the data n is set at 
"1" in step S554. As a result, input data satisfying level LVLn=l is output. 
Input data satisfying the level LVL=1 is, e.g., data having rank "A" in all 
running states, or data having rank "B" or "C" in the running states of back 
UP " "change lane," "turn right/left," "abrupt acceleration," and "normal," or 
data having rank "D" in the running state of "change lane," "turn right/left , " 
"abrupt acceleration, " and "normal:- or data having rank "E" in the normal 
running state. 

Shen^he level LVLn of input data n is "2," the control steps in FIG. 18 will not 
S executiPis apparent from FIG. 11, data having level "2" is: data having rank 
»B r^c" or "D" in^he running state of "abrupt turn" "-^^^P^'^^^^'^" .. 
guidance," or data having rank "D" in the running state of "^^ange lane, "back 
uD " or data having rank "E" in the running state of "turn right/left," "abrupt 
acieleraSon," aid^-route guidance." Since data having level "2" is not urgent 
: r vehicle ' information and the like during abrupt turn or -^--P^.^^^f ' 
no? p rovided to the driver. Such data is returned from step S552 in FIG 18 to 
the initial routine to postpone the output. Note that the postponed data e.g^ 
"J:hicie information, " L released from the standby ^^-^^^/^^^^^Pf Ji^,^^^^^ 
i s no ot her data) when the vehicle state changes from, for instance, abrupt 
break" to "change lane." 



SeJnwhile, when the level LVLn of the input data -/-?;"the processing 
proceeds to step S560 in FIG. 18. Data having level "3" is: data having rank b 
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in the runni 
data is not 



ng state of "abrupt turn," "abrupt break" and "back up." Since such 
urgent, information offer is prohibited and data is cleared. That is, 
in step S560 in FIG. 18, the queue flag IQn of the subject information n is reset 
(also deleted from the queue table in FIG. 9), and audio information is cleared 
in step S562. In step S564, a message informing a driver that audio information 
is cleared, is displayed on the display 8. 

DEPR: 

In a case where there is outputting data (determination YES is made m step 
S522) , the processing proceeds to step S524 where an argument x of the outputting 
data is saved in the register k. In step S526, a value of the level LVLk of the 
outputting data is obtained. 

DEPR: 

If the value of LVLk of the outputting data is "1" in step S528, the processing 
returns to the initial routine. In other words, if the level of the outputting 
data is "1," outputting operation of the outputting data is continued, but 
inputted data is not offered to the driver and is postponed. Herein, outputting 
data having LVLn "1" is data having rank "A" in the running state of "abrupt 
turn," "abrupt break" and "route guidance," or data having rank "C" or higher m 
the running state of "back up," or data having rank "D" or higher in the running 
state of "change lane," "turn right/left," and "abrupt acceleration," or data 
having all ranks in the normal running state. As explained above, outputting data 
having level "1" is urgent information or information which does not disturb the 
driver in any running states. Thus, output operation therefor is continued. 

DEPR: ^ . ^ V 

Herein, a problem arises if input data having rank "A" (e.g. urgent information) 
is inputted while the outputting data having level LVL "1" is being outputted. 
Particularly, it is problematic when the outputting data has rank "B" which is 
lower than rank "A," because the data having rank "A" may be forced to wait. 
According to the level setting in FIG. 11, the time at which the input data 
having rank "A" is forced to wait is when the outputting data having rank "B" is 
being outputted in the running state which is not "abrupt turn" or "abrupt 
break " Therefore, even if inputted data having rank "A" is forced to wait m the 
running state other than the "abrupt turn" or "abrupt break," it would not cause 
any problem. Furthermore, in the first embodiment, inputted data having ranks 
other than rank "A" is not processed to be outputted during "abrupt turn," or 
"abrupt break" by virtue of the operation in step S508 (FIG. 17) . Accordingly, m 
the first embodiment, such situation where data having rank "B" or lower is 
outputted before outputting data having rank "A" during the running state of 
"abrupt turn" or "abrupt break" would not occur. 

DEPR ' 

In a 'case where determination is made in step S528 in FIG. 18 that the outputting 
data has level "2," a flag OUTPk is reset in step S530 to cancel the output of 
the outputting data. In step S532, the flag IQk is set at "1" to make the 
outputting data k in output-waiting state. The data whose output has been 
terminated is next outputted when the running state which has caused the 
termination changes. 

DEPR: , . 

Herein, outputting data having level "2" is: data having rank "D" or higher in 
the running state of "abrupt turn," "abrupt break" and "route guidance," or data 
having rank "D" or higher in the running state of "back up," or data having rank 
"E" in the running state of "change lane," "turn right/left," "abrupt^ 
acceleration," and "route guidance," It is not problematic if outputting of data 
having level "2" is terminated since it is not urgent. 

DEPR : 

FIG 28 shows the state where output operation of data k which is outputted 
during the running state of abrupt turn, is terminated by another inputted data n 
which causes to change the urgent level to LVL=2, and is resumed after the 
running state returns to normal . 

DEPR * 

Meanwhile, referring back to FIG. 18, if determination is made in step S528 that 
outputting data has level "3," the processing proceeds to step S540 where the 
flag OUTPk of the outputting data k is reset to terminate the output operation. 
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Further in step S542, the subject information is cleared, thus will not be 
outputted in the future. In step S546, a message indicating that the data has 
been cleared is displayed on the display 8 to inform the driver. 



DEPR: 

As has been described above, in accordance with a priority rank, running state or 
conflicting state of information, inputted data is returned to the output queue 
(0UTQ=1) , or output operation of data currently being outputted is terminated and 
the data is returned to the output queue. In the case where the data being 
outputted is returned to the output queue, the data is outputted to a driver in 
output routine in step S600 in FIG. 10 (details shown in FIG. 19) . More 
specifically, data in the output queue is read in step S602 in FIG. 19. The 
identification of the information in the output queue is stored in the register 
X. In step S6 04, the number of times of continuous data output is obtained. 
Details thereof will be described later. 



DEPR: . . ^ -^ 

If the number of times of the continuous data output is withm a limited value, 
it is confirmed in step S606 that there is no other data being queued as output 
data, and in steps S608 and S610, subject information x is marked m as cputputted 
data' {OUTPm=l) . Then audio information is outputted-in step S612 to a digital 
signal processor (DSP) (not shown) in the centralized control unit 2 to be output 
as audio information. In step S614, output starting time (TMST) of the data is 
stored in a register. The processing returns from step S614 to the mam routine 
in step S700 (FIG. 10) . 



Upon completing audio information output via the DSP, the completion is detected 
as an interruption. When output- complete interruption is detected, the control 
steps shown in FIG. 20 begins. It is confirmed in step S802 in FIG, 20 that there 
is at least one entry in the queue being outputted. In step S804, the data number 
in the queue being outputted is stored in the register q. In step S806, the 
output queue (OUTP) is reset to indicate that output has been completed. In step 
S808, the time at which the output has been completed (TMED) is stored in a 
register . 

DEPR: ^ . , 

Herein, the routine in FIG. 19 for obtaining the number of times will be 

described . 



DEPR: - . 

When data, particularly audio information, is continuously output, a driver is 
bothered by such information. Therefore, it is preferable to reduce an excessi 
amount of audio information provided to a driver in a continuous manner. Howev 
the number of times of continuous information offering should be regulated m 
accordance with running conditions. Therefore, in the present embodiment, 
limitation in number of times LMT of continuous output is defined as shown m 
FIG. 21. 



DEPR ' 

A driver feels that data is "continuously outputted," when the time difference 
between output -completed time (=TMED) of data and output starting time (=TMST) of 
the next data is small. Assuming that a threshold value of the time difference is 
set to a predetermined value .delta., it is defined herein that a driver feels 
uncomfortable when a sequence of data output satisfying TMST-TMED< . delta . is 
repeated for the number of times LMT. In the present embodiment, when data is 
outputted the LMT number of times, data waiting in the output queue is removed 
from the queue, so that the driver will not be bothered by continuous output of 
audio information. Note that the limitation in number of times LMT varies in 
accordance with the running state. Therefore, in the first embodiment, the ^ 
limitation in number of times LMT is varied in correspondence with the running 
state as shown in FIG. 21. Since data should not be removed while a vehicle 
performs urgent operation (abrupt turn or abrupt break), the LMT value is not 
limited under such condition. On the contrary, under a driving state which is 
closer to the normal driving condition, a driver drives in free and easy manner. 
In other words, the driver is more tolerant of information offer. Thus, larger 
value of LMT is set. 



DEPR: 
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The number of times may be obtained at the time of starting output operation. 
More specifically, step S604 (details shown in FIG. 22) is executed each time the 
output routine in FIG. 19 is executed. In step S572 in FIG. 22, TMST-TMED is 
calculated. In step S574, it is determined whether or not the difference obtained 
in step S572 is larger than the threshold value .delta.. If it is larger, the 
system judges that the continuous output so far is not problematic. Then in step 
S582, a counter CNTR which stores the number of times of continuous output is 
reset . 

DEPR: 

Meanwhile, if the difference exceeds the threshold value .delta., the processing 
proceeds to step S576 where the counter CNTR is incremented. In step S578, the 
counted value CNTR is compared with the limitation in number of times LMT 
corresponding to the current running state STAT. If the counted value CNTR 
exceeds the LMT, a data output request (IQ=1) which is queued in the output table 
is searched in step S580. Then the queue of data output is reset in step S590. 

DEPR: 

Furthermore, the urgent level LVL indicative of the timing of data output is 
determined on the basis of both the running state STAT and priority rank RK of 
data as shown in the tables in FIGS. 11 and 12. Accordingly, uniform information 
offer, where data output is determined based on only the running state STAT or 
only the priority rank RK, is prevented, and well-balanced information offering 
is realized. This is because the timing at which information is provided to a 
driver should be determined by taking account of both nature of the information 
(rank, that is, importance) and current running state {urgent level ) . 

DEPR: 

Moreover, by virtue of the fact that continuous data output is limited as shown 
in FIG. 21, a driver will no longer be bothered by an enormous amount of 
information being offered. In addition, the limitation in the number of times of 
continuous output is changed in accordance with the running state. Thus, a 
situation such as where important information is lost depending on the running 
state, does not occur. 

DEPR: 

Hereinafter, description will be provided in the first modified example where 
modification is added to the controlling operation (FIGS. 10 to 28) of the 
foregoing first embodiment. According to the first embodiment, when audio 
information having rank "B" (information related to vehicle ) is inputted in the 
running state of abrupt turn or abrupt break, LVL=2 is given to the data because 
data having rank "B" is considered as relatively important. Thus, depending on 
conflicts with other information, the audio information having LVL=2 is provided 
to the driver. 

DEPR: 

FIG. 67 is a setting table where urgent levels of output timing are set, which is 
used in controlling of the first modified example. The table in FIG. 67 differs 
from the table used in the first embodiment (FIG. 11) in that, as described 
above, LVL=1 is given to data having rank "B" in the running state of abrupt turn 
or abrupt break, and the data is offered to the driver always on a display. 

DEPR: 

The first modified example employs substantially the same control steps described 
in the first embodiment. However, the routine "processing corresponding to 
priority" (FIG. 18) in the first embodiment is substituted with control steps 
shown in FIG. 68. More specifically, in the controlling of the first modified 
example, when data having level LVL=1 is detected while the data has rank "B" and 
when the running state of the vehicle is abrupt turn or abrupt break, the 
inputted data is converted to text data (or kept as text data) to be displayed on 
the display 8 . 

DEPR: 

Note that in the first modified example, the information having rank "B" (e.g. 
vehicle information) is regulated so that it is not provided as audio 
information. However, during the running state of abrupt break or abrupt turn, a 
driver may feel annoyed even when audio information having rank "A" is provided. 
To cope with this, the first modified example may be modified further such that 
audio information having rank "A" is converted to text data to be displayed. 
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DEPR: 

In the above-described first embodiment, the timing of information offer, 
particularly in a case where two conflicting information are simultaneously 
inputted, is determined by introducing the concept of the urgent level LVL (FIG. 
11) . The urgent level LVL is determined on the basis of the priority rank RK and 
running state STAT as shown in FIG. 11. 

DEPR: . . ^ 

FIG 29 shows a table for determining priority, while taking into account ot 
running environment ENV for each type of information. The priority in the second 
embodiment indicates a classification of control action which is to be taken 
after the priority is determined. To be discriminated from the "level LVL" used 
to determine priority in the first embodiment, the priority in the second 
embodiment will be referred to as "classification CL" for descriptive 
convenience. In the second embodiment, three types of running environment are 
set: a mountain road, neighborhood and a congested road. According to FIG. 29, 
the same type of information has different levels of importance to a driver 
depending on a running environment. Thus, depending on the change m running 
environment, a value of priority, i.e. classification CL, is different. The 
smaller the value of classification CL, the more the information is important for 
a driver For instance, NAVI information has high importance (value "2") in a 
congested road, but has medium importance (value "3") in a mountain road, and has 
low importance (value "4") in the neighborhood since a driver is familiar with 
the area. Note that in the second embodiment, data having the classification 
value CL "1" is always outputted, but data having classification CL "4" is 
excluded from output object. 

DEPR ' 

More specifically, by the time the processing in step S700 in FIG. 31 is 
executed, the number x of the data, subjected to information offer, has been 
registered in the register n. Thus, in step S700, it is determined before 
processing data n, whether or not there is data x currently being outputted (for 
descriptive convenience, will be referred to as "outputting data" as similar to 
the first embodiment) . 

DEPR ■ 

Description is now provided for a case where the classification CL of the 
inputted data n is "3." CL=3 denotes that a level of importance of the inputted 
data n is relatively low under the current running environment. Therefore, it is 
not necessary to stop output operation of the current-outputting data k having 
hiqh priority rank RK. It is even preferable not to output the inputted data n, 
considering the fact that the priority rank of the outputting data k is higher 
than that of the inputted data n, and the level of importance of the inputted 
data n is relatively low under the current running environment. It is preterabie 
not to output the data n because the system can prepare for another important 
data which may be inputted next. Thus, the input queue flag IQn of the inputted 
data n is reset in step S722, the data is deleted in step S724, and a message 
indicating the deletion of the data is displayed in step S726 . FIG. 34 
illustrates the above description. 

DEPR * 

When a priority rank RK of outputting data k is lower than inputted data n, 
output operation of the outputting data k is terminated depending on a condition. 
Whether or not the output operation is to be resumed is determined in accordance 
with the running environment. More specifically, when two conflicting data exist, 
whether or not to stop output operation of current-outputting data is determined 
in accordance with priority ranks between the conflicting data (step S704) . 
Whether or not to resume the output-operation which has been terminated is 
determined on the basis of the level of importance (classification CL) of the 
data, while taking into account of the running environment (steps S708 and S710) . 
When it is determined that the terminated output operation will not be resumed, 
the data is cleared (step S714) . 

DEPR * 

Meanwhile, when a priority rank RK of outputting data k is higher than inputted 
data n (step S720) , whether or not the inputted data n is to be provided to a 
driver is determined on the basis of the level of importance (classification CL) 
of the inputted data n, while taking the running environment into consideration. 
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In other words, output operation of the outputting data Jc is maintained, but 
inputted data n having lower priority rank is regulated in accordance with the 
level of importance of the inputted data n in the running environment {steps 
S722-S724) . 

DEPR: 

According to the third embodiment, when there are a plurality of data inputted 
simultaneously to be provided to a driver, whether or not to provide the driver 
with the data, and the priority order of information offer are determined in 
accordance with a running mode . 

DEPR: 

In the third embodiment, whether or not to provide information, and the p riority 
order of the information offer are determined in accordance with a running mode, 
that is, a general driving mode ( low- to-medium speed) and a highway driving mode. 
As shown in FIG. 35, priority PR and classification CL corresponding to the 
running mode are defined to each type of information. More specifically, 
depending on the running mode, different priority values PR and different 
classification values CL are given to the same information. For instance, NAVI 
information has priority PR=3 and classification CL=2 in the general driving 
mode, but has PR=6 and CL=3 in the highway driving mode. Since a driver is less 
likely to lose his/her way on a highway, the NAVI information is less necessary 
while Vies information is necessary on a highway. These settings can be changed 
by a user as similar to the first embodiment. 

DEPR : 

The determination logic of the order of information offer is basically the same 
as that of the second embodiment. Particularly noted herein is a logic of 
determining whether or not to provide the information to a driver. In a case 
where priority PR of outputting data k is lower than inputted data n, output 
operation of the outputting data k may be terminated. Whether or not the 
terminated output operation will be resumed is determined on the basis of the 
running mode. In other words, when there are two conflicting information, whether 
or not to terminate output operation of current-outputting data is determined on 
the basis of priority of the conflicting information (step S1824 in FIG. 42) . 
Whether or not to resume the terminated output operation is determined on the 
basis of a level of importance (classification CL) of the information, while 
taking the running mode into consideration (steps S1828 and S1860) . In a case ^ 
where it is determined that the output operation will not be resumed, the data is 
cleared (step S1864) . 

DEPR: . , , . ^. u 

Meanwhile, in a case where priority of the outputting data k is higher than 
inputted data n (NO in step S1824) , whether or not the inputted data n should be 
provided to a driver is determined on the basis of the level of importance (CL) 
of the inputted data n, while taking the running mode into consideration. In 
other words, the output operation of the outputting data k having a higher^ 
priority than inputted data n is ■ maintained, while the inputted data n having 
lower priority than data k is regulated in accordance with the level of 
importance of the inputted data n in the current running mode (steps 
S1834-S1838) . 

DEPR • 

The third modified example determines the order of information offer and whether 
or not to provide the information to a driver in accordance with the time zone 
where driven. 

DEPR * 

FIG 48 shows a definition table of priority PR and classification CL as similar 
to FIG 35 described in the third embodiment. Values of the priority PR and 
classification CL are defined (but can be changed) for each type of information 
as similar to the third embodiment. Even for the same information, different ^ 
values of priority PR and classification CL are given depending on difference m 
time zone . 

DE PR • 

According to the third modified example, the time zone may be designated by the 
name (e.g. meal hour, commuter's rush hour etc.) as shown in FIG. 48, or by 
specifying the time as shown in FIG. 49, 
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DEPR: 

The fourth modified example is a modified version of the third modified example, 
and the determination process of whether or not to provide information to a 
driver, is further improved. According to the foregoing third modified example, 
data which is excluded from the determining operation of information offer is 
predetermined {indicated by "x" in FIGS. 48 and 4 9) in the system, and the 
setting can be changed by a user via the display. However, information subjected 
to processing is different in accordance with the time zone. Therefore, the 
fourth modified example includes a table shown in FIG. 50. In the table, 
information are determined in advance as excluded from the determining operation 
of whether or not to provide a driver, by the driver, and the determination is 
made in accordance with the time zones. 

DEPR: 

More specifically, upon executing step S18 06, driving time is read and the 
driving time zone is determined based on the time. Examples of the determination 
logic are shown in FIG. 52. By utilizing the logic, the current driving time zone 
can be determined without requiring additional user operation. Upon determining 
the driving time zone, data to be excluded from the determination operation of 
whether or not to offer information, is decided by the logic shown in FIG. 50. 
Note that in FIG. 50, information without the indication . smallcircle . is the 
data to be excluded. 

DEPR: 

The control operation performed by the fifth embodiment utilizes the same control 
described in the third embodiment. FIG. 56 is a table where levels of importance 
(priority PR and classification CL) of each type of information are defined in 
accordance with driver's mental state. The table in FIG. 56 is utilized in the 
fifth embodiment in the manner similar to the way the table in FIG. 35 is 
utilized in the third embodiment to perform controlling in accordance with 
running state. More specifically, for the control steps of the fifth embodiment, 
step S1806 in the flowchart in FIG. 38 is changed to step S1806 • as shown in FIG. 
58. In step S1806», the table in FIG. 56 is referred to. Note that in the fifth 
embodiment, the system determines the mental state of the driver on the basis of 
continuous driving time, the driving time zone, the number of times of breaking 
and driving speed and so on. 

DEPR: 

In the control performed by the third embodiment, so called "urgent information" 
is processed as exceptional data in the routine "processing exceptional data" 
{FIG, 38) , so that the urgent information is quickly and surely provided to a 
driver. However, such "urgent information" in the third embodiment is data which 
has been predetermined by the system. Examples of "urgent information" are: 
tunnel information, earthquake information, vehicle information (alarm for 
breakdown, alarm for distance between vehicles, alarm for flat tire and the 
like) , traffic information (accident information, regulation of traffic) and the 
like. Thus, "urgent information" is fixed. The sixth embodiment is characterized 
in that the "urgent information" can be designated by a driver. 

DEPR: 

As described above, sixth embodiment handles information designated by a user as 
"urgent information" in addition to the innate "urgent information." As modified 
example of the sixth embodiment, for instance, the following process may be 
performed. That is, difference is calculated between the priority order of the 
innate urgent information and the priority order of information in question. If 
the difference is less than a predetermined value (e.g. 2), the information in 
question is forcibly set as "urgent", as shown in FIG. 77. 

DEPR: 

Control performed by the seventh embodiment utilizes most of the control steps of 
the third embodiment. FIG. 62 shows only the steps different from the control 
steps of the third embodiment. By adding the control step S2 000 (FIG. 62) between 
steps S1804 and S1806 (FIG. 38), the control operation of the seventh embodiment 
is realized. By such control steps, the system according to the seventh 
embodiment performs, in step S2000, correction of the priority order which 
satisfies the rules in FIG. 61, on the basis of various data read in step S1804 
{FIG, 38) . In step S1806, classification value CL and the limitation in number of 
times of continuous output LMT are read in addition to the priority PR corrected 
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in step S2000. Then in step S1810, the routine "processing corresponding to 
priority" is performed in accordance with the correction value. 

DEPR: 

Conditions of correction in FIG. 61 are as follows: correction to raise the 
priority order of vehicle trouble information in a case where a problem suddenly 
occurs in the vehicle, correction to raise the priority order of VICS information 
while driving a highway, correction to raise the priority order of weather 
information while driving a mountain road (detected by a barometer) or driving in 
rain (detected by wiper being activated) , correction to lower the priority order 
of information having high -level noise or information which has passed for a long 
time since the time of input, correction to lower the priority order of VICS 
information related to directions different from destination, and the like, 

DEPR: 

More specifically, during route guidance, the navigation controller of the 
navigation system notifies the driver each time the vehicle approaches one of 
plural route guidance points, so that a driver is able to confirm that he/she is 
correctly heading toward the destination. In other words, the driver expects to 
receive some amount of information near the route guidance points. However, in 
locations other than the route guidance points, the driver does not expect any 
information, thus feels uncomfortable when provided with excessive amount of 
information . 

DEPR: 

To cope with such situation, the navigation controller 17 according to the 
seventh embodiment sends NAVI information to the centralized control unit of the 
seventh embodiment in accordance with the control steps shown in FIG. 78, while 
adding a priority order to the NAVI information. As shown in FIG. 78, the NAVI 
controller 17 confirms whether or not route guidance control is being performed. 
If the route guidance control is performed while the vehicle is near (within a 
radius of a predetermined distance) one of the route guidance points, the 
priority order of the NAVI information is raised. 

DEPR: 

Meanwhile, with respect to other information sent by a medium besides NAVI 
information, if the route guidance control is performed while the vehicle is near 
one of the route guidance points, the priority order of the information, e.g. 
news or the like, may be lowered, 

DEPR: 

Accordingly, NAVI information is prioritized when a vehicle is near the route 
guidance point, and priority orders of information other than NAVI information 
are lowered. Therefore, information truly necessary for a driver is provided. 

DEPR: 

Depending on running speed of a vehicle, information such as traffic information 
or the like may be inputted many times while driving a short distance. Such 
repetition of input may bother a driver. 

DEPR: 

To cope with such situation, the seventh modified example lowers the priority 
order of data which has the same contents as that of already- inputted data. As a 
result, a situation where the same information is repeatedly offered to a driver, 
is prevented. The seventh modified example is particularly effective for data 
transmitted in the form of character or text because such data is relatively easy 
to make determination of its contents. By lowering the priority order of the 
character data or text data having outdated contents, probability of such data 
being provided to a driver (on a display or as audio information) will relatively 
decrease. Furthermore, since the multimedia navigation system outputs character 
data by converting the data into audio information, restraining such outdated 
information from being displayed or being outputted as audio information, will 
reduce the driver's uncomfortable feeling. 

DEPR: 

The seventh modified example also employs a part of the control steps of the 
third embodiment. More specifically, control steps shown in FIG. 64 is added 
between the steps S1804 and S1806 of the control steps shown in FIG. 38. In^ 
addition, a storage area as shown in FIG. 63 is prepared in the seventh modified 
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example. In the storage area, time at which character data is inputted, the 
number of times the data is determined to have the same contents, and the 
contents of the data are stored. 

DEPR: 

It is determined in step S2002 in FIG. 64 whether or not inputted data is 
character data. If the inputted data is not character data, the inputted data 
will not be subjected to determination of whether or not it is outdated. If the 
inputted data is character data, it is determined in step S2004 whether or not 
the storage area in FIG. 63 includes data having the same contents as that of the 
inputted data. If the storage area does not have any data having the same 
contents, the inputted data is stored in the storage area in FIG. 63, If data 
having the same contents is found, the time difference is calculated in step 
S2008. If the time difference is smaller than a predetermined threshold value 
.DELTA., it is determined that data having the same contents has been received in 
a short period of time. Therefore, the priority order of the data is lowered in 
step S2012, Meanwhile, if the time difference is larger than the predetermined 
threshold value .DELTA., it is determined that a long period of time has passed 
since last received the data. Therefore, the data is deleted from the storage 
area shown in FIG. 63. 

DEPR: 

Note that in a case where data having the same contents is inputted for the 
second time, the data may be displayed on the display 8. Further, when the same 
data is inputted for the third time, the data may be deleted. 

DEPR: 

In a case where audio information is frequently provided to a driver, the 
interval of audio information output is also an important factor of driver's 
concern. Thus, the system according to the ninth embodiment controls the interval 
of information offer. According to the present embodiment, a long interval of 
information offer is set for conditions (running environment, running state, 
driving time, mental state) where a small amount of information offer is 
preferable, and a short interval of information offer is set for the conditions 
(running environment, running state, driving time, mental state) where a large 
amount of information offer is preferable. By virtue of the above, the amount of 
information offer is controlled to a most appropriate amount in a unit of time. 

DEPR: 

Referring to FIG. 70, time T denotes the time interval (minimum value) of 
information offer which is most appropriate for a driver in the current 
condition. The beginning of the time interval of time T is the output - completed 
time (TMED) of the previous data. Therefore, the driver will not feel 
uncomfortable if the next data is provided after the lapse of TMED+T, 

DEPR: 

Generally speaking, the time interval should be shortened as the vehicle speed 
rises. However, in a case the vehicle safely runs on a highway, even though the 
vehicle speed is high, it would not be so problematic to provide a large amount 
of information. Thus as shown in FIG. 71A, the value of tO is set small when the 
vehicle speed is above 8 0 km/h . 

DEPR: 

Referring to FIG. 71F, the correction coefficient t5 is set as a larger value as 
the limitation LMT in number of times of continuous data output (see FIG. 21) 
becomes larger. As apparent from FIG. 21, the limitation in number of times LMT 
is set as a smaller value as the driving condition requires more operation to the 
driver. The more the driving state is close to the normal state, the less 
information may be provided. 

DEPR: 

As explained with reference to FIG. 70, when output of the previous data is 
completed, a delayed time T (interval of information offer) is calculated in step 
S2210 in FIG. 73 in accordance with FIGS. 71A to 71F. In step S2212, the time T 
is set at a timer . 

DEPR: 

Then, when audio information (e.g. data k) is inputted, the determination control 
of whether or not information is to be offered to a driver and the priority order 
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of information offer, described with reference to the first to eighth 
embodiments, is performed. Then, an output queue (OUTQk=l) is set for the audio 
information k (e.g. step S554 in FIG. 18). Referring to the "output" routine in 
FIG. 72, it is determined in step S2200 whether the audio information k is urgent 
information (KND^l) or whether the current position is near a route guidance 
point (e.g. junction) of the navigation system. This is because output of urgent 
information (KND=1) should not be delayed, and because the driver necessitates a 
larger amount of information near a route guidance point. 

In a case where the audio information k is not urgent information and the current 
position is not near a route guidance point of the navigation system, it is 
determined in step S2202 whether the time set at the timer in step S2212 (FIG. 
73) is up. If the set time has not passed, it is too early to offer the audio _ 
information k to the driver. Therefore, the output queue of the audio _ information 
k is reset (OUTQk=0) in step S2204. By resetting the queue of the audio 
information k, the system becomes available to output another data having higher 
priority which may be inputted next . 

DEPR * 

In the ninth embodiment, the timer T is started each time output of the previous 
audio information is completed. According to the eighth modified example, the 
timer T is started each time the output of the previous audio information is 
started. In this case, the time necessary to output audio information needs to be 
calculated as shown in FIG. 74. 

FIG 75 shows as an example a display screen of a user interface where the form 
of audio information output is set. In the tenth embodiment, sound quality and 
volume are set according to types of guidance information: "warning information," 
"route guide information," "traffic information," "facility guide information," 
"highway guide information," "current position information," "running 
environment," "driving time, " "driving time zone" and "driving area." With 
respect to a type of information which requires driver's attention, tense voice 
is set and information is provided in a tone of command. On the other hand, in a 
driving condition where the driver should be relaxed, for instance, soft female 
voice is set and information is provided in a tone of guidance. 

DE PR * 

FIG 80 is a logic table ("table for determining audio information output 
destination") for determining the medium to which audio information is to be 
outputted on the basis of the "urgent level" which is determined based on the 
running state and conditions, and the "priority rank RK" of the data itself - 
eleventh embodiment is characterized in that a user is able to select the 
determination logic. FIG. 79 is a table ("definition table of menu-setting 
permission") defining the areas where a user is allowed to take part m 
determination . 

DE PR * 

Hereinafter, the urgent level in the eleventh embodiment will be described. 
DEPR * 

The urgent level is, as similar to the foregoing embodiments, an index to express 
a level of urgency of information based on various aspects other than the 
infHTSJStion itself. In the present embodiment, the urgent level is defined as 
shown in the table in FIG. 81. It is defined that the smaller the value of the 
urqent level, the higher the urgency. Note that most states shown in FIG. 81 are 
determined by various sensors shown in FIG. 2. For instance, "prediction of large 
curvature on the road" in the first column of the table in FIG. 81 can be 
determined on the basis of information sent by the NAVI controller, e.g. 
information notifying that the vehicle is approaching a curve. 

DEPR ' 

Referring to the table in FIG. 80, the sign X denotes that information is not 
offered. According to the "table for determining audio information output 
destination" in FIG. 80, the lower the priority rank RK (indicative of urgency of 
data itself) is, and the higher the urgent level as a running state (vertical 
column of the table in FIG. 80) is, the necessity to provide such data is low. 
Thus such data is not provided to the driver as audio information or not 
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displayed on a display. For instance, when priority rank RK=E and urgent level 
LVL=1, data is not outputted. On the other hand, the higher the priority rank RK 
of data itself is, and the lower the urgent level as a running state is, the 
necessity to provide such data is high. Therefore, such audio information has a 
wider selection for output destination. For instance, when priority rank RK=A or 
B and urgent level LVL=4 , the data can be outputted as audio information or 
displayed on a display (display 8 or head-up display) . 

DEPR: 

As set forth above, the output destination of audio information is selected in 
accordance with running state of a vehicle . Accordingly, appropriate information 
can be provided to a driver in appropriate timing . 

DEPR: 

For the descriptive convenience for describing the operation of the twelfth 
embodiment, it is assumed herein that the navigation system of the twelfth 
embodiment receives the following types of information: "warning information," 
"route guide information," "facility guide information," "highway guide 
information," "traffic information" and "current position information." These 
information are "guide information" for the driver. Environment conditions which 
become the base of these information is detected by various environment detecting 
means exemplified in FIG. 83. When warning indicative of a small amount of 
remaining fuel is issued, what the driver needs is a series of guide information 
to deal with the warning. FIG. 84 shows examples of such guide information. In 
other words, it is preferable for the driver to be provided with the guide 
information in a logical series of sequence to deal with the warning. More 
specifically, if facility guide information is provided before warning 
information, the driver must take time to figure out why the facility guide 
information is being offered. It is difficult for the driver to think such matter 
while driving. In the navigation system of the twelfth embodiment, the order of 
information offer is designated by using a user interface shown in FIG. 85. In 
FIG. 85, the priority order of information offer is specified by a user. Thus, 
the designation of information offering order designated by a user can be applied 
to the navigation system of the aforementioned first to eleventh embodiments 
since all the navigation systems utilize the "priority order." While the^ 
navigation systems according to the first to eleventh embodiments determine the 
priority order on the basis of a level of importance or a level of urgency, the 
system of the twelfth embodiment determines the priority order such that a 
sequence of information offer makes sense to the driver. 

DEPL: 

Herein, the time tO is the basic value of information offer interval which is 
determined according to vehicle speed. 

DEPC: 

Third Modified Example (Considering Time Zone) 
DEPC: 

Fourth Modified Example (Considering Time Zone) 
DEPU: 

C: Occurrence of facility guide information is determined by whether the vehicle 
runs near a destination point or route guidance point while route guidance is 
being provided; 

DEPU: 

D: Occurrence of highway guide information is determined by whether or not the 
vehicle is running on highway; 

DEPU: 

E: Occurrence of traffic information is determined by whether or not there is 
congestion or accident on the road the vehicle is running; 

DEPU: . ^ T. T. u 

F: Occurrence of current position information is determined by whether or not the 

vehicle is running an area where driving frequency is low. 
DEPV: 

"-1: Information offered to a driver is regulated by not providing information 



17 of 18 



9/11/01 9:00 AM 



TR.ecord ETisplay Form 



http://westb^s:8820^in/gate.exe?f==doc&s...e=&p_Message=&p_doccnt=l&p_doc_l-PTFKWIC 



having low priority (regulating data output by RKLMT shown in FIG. 13, or 
prohibiting output of data having level 3 as shown in FIG. 12) . 

4, The navigation apparatus according to claim 1, wherein the priority order is 
prescribed in accordance with a running state of a vehicle . 

CLPR * 

5. The navigation apparatus according to claim 4, wherein the priority order is 
changed in accordance with change in the running state of a vehicle . 

CLPR * 

6, The navigation apparatus according to claim 4, wherein the priority order is 
prescribed in accordance with running environment of a vehicle . 

CLPR • 

7. The navigation apparatus according to claim 6, wherein the priority order is 
changed in accordance with change in running environment of a vehicle. 

CLPR * 

17 The navigation apparatus according to claim 1, wherein the running state is 
inclusive of a driving time zone during which the driver is driving, and said 
allocating means sets the priority order and travelling classification of 
inputted information to a value according to the driving time zone. 

CLPV • 

means for allocating, to inputted information, a rank indicating a type of the 
inputted information, and a priority order indicating a priority degree of the 
inputted information; 

means for setting an urgency level indicating a degree of urgency of the inputted 
information with respect to travelling, the urgency level being set to either one 
of a highest first, intermediate second and lowest third values in consideration 
of the allocated rank and at least one of running state, driving purpose and 
running environment; and 

control means for allowing a provision of the inputted information suspending the 
provision of the inputted information until the provision is allowed, and 
canceling the provision of the inputted information when the set urgency level 
indicates the highest first, intermediate second and lowest third values, 
respectively . 

means for detecting a time interval between the audio information successively 
provided by said providing means; 

restraining means, operatively connected to said detecting means and said 
providing means, for restraining said providing means from providing the audio 
information successively, if it is detected that a predetermined number of 
conditions have successively occurred in the time interval less than a 
predetermined time length. 

means for allocating a priority order to inputted information in accordance with 
a type of the inputted information, the priority order being prescribed in 
accordance with a quality of information; 



CLPV 

mean^ ^- _ . 

priority order of the selected type of information, 



means for providing the driver with the selected information in accordance with a 
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